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are the fcllowingt. Do eniplcyers actually select on the i^sLs cf 'rz^e 
competencies they desire in job applicants? How do selecrion 
practices vary according to the type of job and level cf skill 
required entry? What biases enter into selection_decisions as a 
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effectiveness of their selection procedures? The chapter concludes 
with a discussion of how organizational realities and ether factcrs 
affect selection practices and what educators can iearn from curr^ 
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I. INTRbbUCTlbN 



. In a competitive economy where more than one applicant may 
be found for most jobs or career opportunities that are made 
available, there exists their need for mechanisms to select the 
most qualified applicants out of the rest of the competition. 
From an employer's point of view, a selection procedure shoulc 
be able to identify people who are right for the job as well as 
to ensure that selected individuals will choose to remain in 
the job long enough to justify the employer's investment in 
training. The degree of success in meeting these aims' varies 
greatly with the selection procedure usee and with the require- 
ments of the job to be filled. 

Nearly all the selection procedures that are currently used 
were in use more than 30 years ago. Today, many of these 
procedures evidence a greater degree of sophistication in their 
design and application than that which was tolerated in a more 
naive age. Thirty years ago, subjective judgments on the part 
of the interviewing employer, a bachelor's degree from a "good 
school", and test scores on a variety of psychometric measures 
of general ability all had a life of their own; the relation- 
ship between these factors and the merit of applicants for 
available jobs . remained, with few exceptions, largely unques- 
tioned outside of academic circles. Indeed, very few studies 



o£ the cost-effectiveness of tiese dev i^^s undertaken 

because of their appearance S valifiii^y a:.d ^«:ause the costs 
of such studies in both> dollars and :=^C}nxT^ .:srce outweighed 
the costs of administering th^ selecrdr^n ^;^£t:sirs themselves* 

Title VII of the Civil RrcSts Ac" 19^ ::sheired in a new 
era and a new conscicasness wzrh regarc ^-'^ eaployee selection. 
Title VIX is the portion of the Civi_ ' .qhts ^t that deals 
with discrimination im emplovm^t. I is e^^cially relevant 
to employment selection processes ber^ ~ trf Section 703 (h)# 
known as the Tower Amendment, wfaicn sst^es ^>at it is not "an 
unlawful employment practice for si ^'^plco^er to give and to act 
upon the results of any profession?; 1 1; ^cevel^ped ability test 
provided that such test, its aamirtlstz^i^Lor rr action upon the 
results is not designed, intended cr ^c? discriminate be- 

cause of race, color, religion, sex atimial origin*" This 

section was added to the Act as a r erf concern in Congress 

that companres would be prevented by - .ouj^ts from using em- 
ployment tests which disqualified i&ec ^r- ^ protected classes 
(i.e.,. those identifiable by race, . z, religion, sex, or 
national origin) . The phrase "prof . :)nally developed test" 
became the impetus for the 1970 ver - cr the SEOC selection 
guidelines. The guidelines served rzz :e£ine what a test is, 
what constitutes dirjcriminatory use tests, and what stand- 
ards of validity should be used to ji^e whether a test has 
been developed carefully, fairly, and .igorousiy. These 
guidelines, in turn, became the touchsz:Lone , implicitly and 
explicitly, for professional practice ^d legal decisions. 



Employers were now compelled to demonstrate statistica^r . y 
or rationaJ—ly the relaticnship between selection procedures and 

jobs, to develop pxoceSrres which met propcaed standards of 

validity, and to c^reSuxl:* docunsr.t the fairness of -diese ^^r:::^ 
cedures to previously ?rr?tBCted groups. Many orgahization£ 
began to shj^ away f ronr ^es±:ir:^ e-rfaer out cf an inability r 
deal wis t±= cost ana rt^oi c i ralidation cr out of the r^li- 
zation that rheir procedures <' ic not measur - up to the guide- 
lines. In 1972, when air. srce d iu^L to Title VII ga'^e the EEDG 
enforcement power, the certain selection procedures 

abated even more swiftly. Iron^ally, psychometric tests of 
ability and personality, tz±rose selection devices for which 
statistical data were mo^rr readily available^ were the first 
devices to be abandoned, since they were aiso the first to -^ow 
^ discriminatory impact. On the other hand, other devices such 
as the selection interview, while generally much lower in reli- 
ability and validity, tsve increased in use, since it is more 
difficult to document ^-e adverse impact of such devices. 

At the same time, — e last ten years have etched a growing 
concern in the minds of employers regarding the efficiency and 
productivity of the wor-: force. ?ierce competition from 
abroad, where labor was Less costly and where technology had 
caught up wi-th the domestic standard in many areas, contributed 
initially to this concern^ In more recent years, the slowing 
of economic growth due tc currency inflation, the increasing 
cost of energy, ano a cJerfLine in the availability o£ resources 
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nave placed accdtiohal eiitphasrr. ipoe producti-ritv as r=he key 
issue for the American economr^ T*ne recent consumer urovemeht 
has created iis own pressures -assuring tb^ value and gual- 
::ty of goods an3 services, and Tr= passage cf the Consumer Pro- 
jection Act n 197.4 placed add:L^^jDnai costs rr: many organiza- 
tions which, in the short run, ^eE3acer±)ated tr e problesi of mafli- 
tia.ning cost3 for goods and s ej: I rres aeliv6-z=r. These sources 

p ressure have placed an adGir:L-'anal burder crn personnel selec- 
tion systems to be cost-effective means of sheeting competent 
inc productive individuals, and to screen our those whose lower 
rrroductivity and shorter tenure make them poor hiring risks. 

The employer of today is faced -with a critical dilemma, 
.ven. the state of the art of psychological measurement, is it 
possible to use a selection procedure that enhances the ability 
an organization to employ individuals whc are both competent 
and highly productive, v/hile not excluding disproportionately 
individuals on the basis of characteristics such as age^^race, 
or sex which are, in theory, not correlated with job perform- 
ance? Have employers pursued advancing the art and the practice 
of selection to make selection both more fair and more predic- 
tive of effective performance? Gr have they turned away from 
selection systems as devices for ensuring competence and toward 
on-the-job training and development? 

This chapter is devoted to identifying the state of the 
practice in competence assessment in employer selection. It 



proceeds with* a rev^w of selecticr: irechniques j.n current use, 
with particular emjSssis oh variati-bhs/ validities, and problems 
associated with eacri^ The section that follows shifts focus to - 
'the selection pract±re in more'^han 200 jobs which represent 

th.e spectrum of car»r apportimity. The concluding section 

__ _ _._ _ ___ 

responds to • three cr=ical issues rz:r the practice of personnel 

selection: the cisc=::?ancy between actual and ideal practice; 

some of the factors underlying the discrepancy; and implications 

of the state of the p.:actice for both educators and employers. 

, _____ _ _ _ _ ______ ^ 

These issues are irzirDcuced in greater detail below so that they 

might serve as a conceptual overview to the discussions which 

follow. , ' * 

Discrep^cy Between Actual and Ideal Practice 

Competence asses^eiit for selection can be viewed from two 
perspectives, that of the personnel researcher, and that of the 
personnel practitioner in a f v/r^irtioning organization. The 
ideals of the former perspective might be tyilikely to converge 
with the realities of the latter. In fact, we expected to find 
considerable discrepancy between the ideal and the actual '^prac- 
tice. This disparity is all the more fruitful for our research 
because it suggests that the selection practices in use today 
can be more controlled, reliable and valid, and that the exist- 
ii:g research oh selection practices can be set as a minimum 
standard for needed improvement. 
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While the research liter*atur_e can be used as a base for 

_ _ /: ' _ 

evaluating the |tate of the practice, what can serve as a stah- 

dard for evaluating competence assessment research itself? 

Although there is ho absolute standard- for a^^sessing the 

quality of resear^rh, over "the years several" researchers have 

written prescriptive documents highlighting the parameters of 

predictors, measures, samples and criteria which wi it lead to 

Optimal selection reliability and validity. These, prescriptions 

remain appropriate for evaluating the state- of the practice 

today. In addition, research that neglect^^^^^hese"^^ - 

caveats in the designing of studies is open to criticism. Com- 

' ' _ \. ( 

parisons between research and praptice# Vith regard to accepted 

standards of excellence, therefore, will be implicitly and 

- - - - - - V _ _ _ _ _ _ _ _ _ ^ ^ 

explicitly stated throughout this chapter * 

ft- 

Factors Underlying the Discrepancy 

As already noted, competing pressures on employers and the 
varying efficiency of employee selection procedures are ex- 
pected to have a telling effect on the practice of competence 
assessment. Other factors which will- later be brought into the 
.picture include the difficulty and practicality of conducting 
studies bn the validity of selection procedures within organiza- 
tions and characteristics of the organizations themselves that 
contribute to the likelihood that validity studies will be 
undertaken. There is always a period of delay between any 
process innovation and its application, and discussion will be 



devoted to an explanation of factors contriUgning to.*delay in 
the- use of state^-of-the-art employee selection devices i 

e» ' • _____ 

_ . Indeed, some 6£ the questions that bear oh impediments to - 

assimilating new assessment technology are relevant to imple- 
menting any new technology* (1) Is the organization ready for 
the innovation? Can it afford a dramatic change in present 
practice, and will top decision-makers in the organization sup- 
port such a change? (2) Is the latest technology really much 
better than -that which is already in lise? Do suspicions of 
change for its own sake create resistar&e to innovation, or are 
the benef its^^Sjf^hange seen as being too marginal to warrant 
it? (3) Do the costs of Implementing the latest technology 
outweigh the benefits to be derived? Regarding employee selec- 
tion, do the costs of validation of a hew selection device 
reduce the likelihoc^3 that a well-documented long-used procedure 

which has wide organizational acceptance will be replaced? 

f~> _ * _ _ _ _ _ ' _ 

These issues flow logically from a discussion of the discrep- 
ancy between ideal and actual practices in employee selection, 
and will be dealt .with oh the basis of data gathered from the 
organizations surveyed in the analysis of job selection 
. ^ procedures, 

\ Implications for Educators and Employers 

As the present study was G^idertaken, it soon became evident 
that exceedingly few employers bas,ed their , requirements for 
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einpibyee selection on competency measurements. Whether or not 
an employer has been able to demonstrate a rational, valid 
relationship between job performance and specified knowledge, 
skills^ abilities or other personal characteristics of the 
individual^ the majority of employers do not attempt to measure 
these^ characteristics directly through an employee selection 
process. Rather, they rely on their ability to make accurate 
inferences' about the presence or absence of job related coiiipe- 
tencies from data collected through devices such as the inter- 
view, personality inventory, or resume. Employers have rarely 
been in the practice of making explicit the competencies they 
seek, except insofar as they are defined by the selection pro- 
cedures they choose to invoke. With the exception of certain 
skill tests and job simulations which provide close correspon- 
dence to the actual work to be performed, most other sources of 
information considered in the- selection process are open to a 
great deal of interpretation in establishing a link between, 
for example, a test score or an event in the employee's back- 
ground and a competency required for the job. 

These observations have certain critical implications for 
both employers and educators. The employee selection proce- 
dures^ in current use are often the only indicators of the 
competencies required for jobs to which educators and job ap- 
plicants have access. It is important^ therefore, to examine 
what, if anything, selection procedures tell us about job re- 
quirements that is either true or misleading. Do employers 
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select applicants on the basis- of competencies that are heeded 
, for satisfactory job performance, and if so, should educators 
better prepare students to survive selection assessment proce-. 
dufes? Or do employers make selection decisions oh the basis 
of competencies that are largely irrelevant to job performance, 
and must educators therefore prepare students for both long- 
term career ef f ectivehess and short-term desirability as a job 
applicant? The answers to these questions will enable us to 
examine the emerging role of the employing organization as ah 
educator of adults, and to contrast that role with the one 
' which is currently being played by the approach to education 

' practiced by a number of contemporary "competency-based" educa- 

tional institutions. 
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11. REVIEW OF EMPLOYEE SELECTION TECHNIQUES 



This section is devoted to selection devices, or screening 
devices that employers apply to job applicants at entry into 
organizations. Promotion and performance appraisal systems^ 
which are used as the bases for selection from within an organi 
zatioh, are hot considered here, though many of the techniques 
described are applicable to this purpose. The interview and 
the psychometric test are the two most popular selection tech- 
niques, and separate sections will be devoted to discussions of 
each. In addition, six other devices-- applicat^ion-blanks-,^^-^ — 
resumes, recommendations, work samples and simulations, thought 
samples, and detectors of deception are currently in widespread 
use and a single section will be devoted to their description. 
Each of the sections will describe different forms of the 
devices, the range of competencies and other factors measured, 
the reliability and validity, of the cJevices, and issues 
associated with their use. . 

Of particular importance in this review is the considera- 
tion of the predictive validity-of a selection technique: to be 
competency-based, a selection device should measure qualities 
of the applicant that are related to effectiveoob performance. 
Presently, employers consider four criteria in determining the 
validity of a selection technique,/ which vary in the degree to 




/ 



which they can be regarded as indicative of employee competence. 
The reader should keep these criteria in mind as he or she 
considers the utility of a selection device for competency 
measurement. 

rl. "Hard" measures, or direct measure of employee per- 
fonhahie^ such as sales volume, production error rate, and port- 
'foiio -prof itability, are to be preferred as criteria, where 
available, against which a selection system would be validated • 

2* "Soft" measures, including supervisor and peer ratings 
of performance, are often collected in the absence of direct 
performance measures. Ratings are susceptible to reliability 
problems and may, in many cases, bear a low relationship with 
harder performance mea^&es (Kane and Lawler, 1979) • 

3. Trairiability , defined in practice as how well an em- 
ployee performs in a training program offered by the employing 
institution, may be a useful criterion for the employer in 
keeping costly attrition low. However, whether traihability 
reflects job competence depends on the to which that. the . 
results achieved through training are stati^stically related to 
job performance. 

4. Attrition, or employee turnover, is a valuable crite- 
rion to the employer .who desires to select employees who ^will 
remain on the job. This criterion is unrelated to competence 
in the job. 

14 . 
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Selection Interviews 



The employment interview has proved to be the most widely 
used technique for employee selection. A survey by Scott, 
et al. (1961) found that 98*4% of the 852 organizations sampled 
used the interview in their selection process. Of the coi^anies 
surveyed 93.9% responded that ah applicant would never be hired 
without first conducting a personal interview. There is no evi- 
dence to suggest that widespread use of the selection interview 
has abated. Campbell, et al., (1970) found that of 1-^6 organi- 
zations surveyed, all placed "great importance" on the interview 
as a selection technique, with only one company attempting to 
decrease the emphasis placed oh it. Perhaps becausfe of such 
widespread use, the selection interview has evolved ihto a 
plethora of forms and styles of which only the most popular 
will be discussed here. 

Variety- in Format 
Structure 

Three primary types of interview structure have been noted 
in the selection interview: the structured, semi-structured 
("guided" or "patterned"), and- the unstructured ("laissez- 
faire" ) .interview. These three approaches differ in the extent 
to which they rely on a standardized set of questions in 
cohauctihg the interview. 
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1. The most prevalent form of the selection interview 
according to Hakel (1977), is the semi-structured or patterned 
approach in which the interviewer covers certain broad areas^ of 
questioning, such as education, work experience, and past 
accomplishments, but maintains discretion over the exact 
phrasing and order of the questions. This approach assures 
that certain data will be gathered from all applicants, but 
does hot assure that comparable data -^ill be gathered from all 
applicants due to differences in phrasings, sequencing of 
questions, and length of time spent on particular areas • This 
is true hot only among different interviewers, but also with 
the same interviewer Questioning different applicants • 

2. The second most popular approach is the "unguided'* or 
"laissez-faire" interview. In this form, interviewers approach 
each interview as a unique situation in which they are not 
bound by areas or questions that' must be covered, but are free 
to pursue those areas that seem to be of most interest. Such 
an approach allows' for a high degree of spontaneity and pro-- 
vides a hign level of interviewer motivation. This approach, 
however, suffers, from a lack of standardization since the same 
questions will not necessarily be asked all applicants for the 
same position. 

3. The least popular approach: in employee selection is the 
structured interview in which the interviewer poses a specific 
set of questions and deviates as little as possible from the 
list. Such a procedure is described by interviewers as being 

16 
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repetitive and monotonous, thodgh it does result in a high 
degree of standardization, thus increasing validity and relia- 
bility of the interview (Schwab & Hennemanr 1969; Carlson, 
et al., 1971; Bass, 1951). 

Administration 

In addition to the format, the interview can differ accord- 
ing to the number of persons conducting it. It is common for 
one interviewer alone to conduct the entire interview with a 
given applicant. However, two other approaches are multiple 
interviews and group interviews. 

1. In multiple interviews/ a number of independent inter- 
views^ with the same applicant are concJucted by different inter- 
viewers. Topics may be divided among interviewers so that 
repetition is held to a minimum. Following the entire series 
of interviews, the interviewers meet together to pool their 
information and opinions and to arrive at a group consensus. 
With such an approach interviewer biases of irrelevant data are 
less likely to affect the final decision. Additionally, gaps 
in information garthered by one interviewer may have been 
covered by other interviewers. Thus, a more complete picture, 
of the applicant's background and characteristics is more like- 
ly to emerge. 

2. A group interview fequifihg two or more interviewers to 
jointly question or be present during the interview of a single 
applicant is relatively uncommon. Interviewers may take turns 
asking questions or one interviewer may question the applicant. 
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while remaining group members observe. As with the multiple 
interview procedure r interviewers meet afterward to make a se- 
lection decision. This process insures that ail inembers of the 
group have the same information on which to base a final deci~ 
sion. with this approach, interviewer biases and irrelevant 
data are less likely to affect the final decision than with the 
more common approach of one interview per applicant. 

Length of Interview 



Despite the fact that the interview is used as a selection 
device by most organizations, there has been little attention 



wide variation reported, rang'ing from less than 15 minutes to 
two hours or. more. Springbett (1958) discovered that the 
average interview lasted about 15 minutes. The average inter- 
viewer was prepared to make a decision about selection after 
only four minuces, while certainly,^ the length of time spent in 
the- interview. would vary according to level of position 
vacancy^ amount of ancillary data available {e...g. , application 
blahits, psychometric test data, recommehdatiohs) , number of 
qualif^ied applicants, and the purpose of the interview (pre- 
liminary screen or actual selection tool) . 




paid' to the optimal length of the interview* There has been 
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I^tstrament Focas 

_•/-'._ 

As might be expected from the wide usage of the interview 
as a selection instrument, the interview has been used to tap 
numerous applicant characteristics, A review by Wagner (1949) 
reported 96 different traits and dimensions that researchers 
have attempted to evaluate by the - interview method. Of these, 
it appeared that the interview was most often used to evaluate 
overall ability, physical appearance, manner, intelligence and 
mental ability, judgment and voice quality. More recent studies 
by researchers have urged that the interview fae used instead to 
measure interpersoi^al relations^ sociability and likeability 
(Otis, Campbell & Prien, 1962; Holt, 1.958; Loevinger , 1959; 
Ulrich & Trumbo, 1965), and job motivartion (Rimland, 1960; 
Woodworth, 1957) since the highest validities are reported in 
these areas ♦ The interview is also a good msisure of 
intelligence, but probably less useful than a psychometric test 
would be to measure mental ability. 

Reliability . There is an abundance of information on the 
reliability of the selection interview* In general, the major- 
ity of information has shown high intirarater reliability. , When 
interviewers evaluate the same applicant: by replaying the 
recorded interview or interviewing the applicant again after a 
peri od— o f time has elapsed, they make ^proximately the same 
ratings as they did the first time (Shtiw, 1952-: Pashalian & 
Cressy, 1953; Anderson, 1954). Thes^ data suggest that an 



interviewer will approach the interview situation similarly 
from one time to the nsort. Hoiwe\rer, when more than one 
interviewer evaluates a "icant, reliability falters. In a 

now classic study, Scot 15) asked six personnef managers to 

'interview 36 applicants sales ability. The interviewers not 
only disagreed oh the fanr.^ng, but for 28 of the applicants, 
the interviewers disagreed oh whether they should t)e in the 
upper or the lower half of the group. Comparable results have 
been found in numerous other studies (Scott, Bingham & Whipple., 
1916; Hollingworth, 1922; Uhrbrock, 1933; Wagner, 1949; Raines 
& Rohrer, 1955; Flag, 1961; Ulrich & Trumbo, 1965). 

One area that has been examined as a Source of interviewer 
^ error is the temporal positioning of favorable and unfavorable 
information. Some authors have reported heavier weighting for 
earlier information, primacy effects, in the evaluation process 
(Blakeney & MacNaughton, 1971), while other authors have re- 
ported heavier weighting of later information or recency ef- 
fects (Fatr, 1973)-^, Peters a Terbofg {1975/ and Tucker & Rowe 
(1979) have concluded that a ^favorable expectancy of the 
interviewer at the start of. the interview followed by he^gective 
information will .result in more favorable ratings than the 
ne gativ "e~expectancy followed by positive ihformatxon. Tucker & 
Rowe explain this phenomenon by suggesting that when favorable 
expectancies exist in the interviewer's mind, he "or she will 
give the applicant less credit for personal successes and hold 
the applicant more personally responsible for past failures. 

20 \ ~ 
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Contrast effects, resulting from the comparison of job appli- 
cants with preceding applicants may also contribute to inter^ 
viewer error (Carlson, 1969; Hakel, bhensorge & Dunnette, 1970; 
Wexley et aL^ , 1972). However r in other studies, contrast 
effects were shown to be minimal (Hakel et al., 1970; Landy & 
Bates, 1973) . . 

In recent years, many investigations have focused on how 
reliabilities can be increased. One such mechanism for height- 
ening interrater reliability is by providing the interviewer 
with more information about the job to be filled* Langdale & 
Weitz (1973) reported that interviewers who were provided with 
30b information about the position they were filling, had high 
interrater reliability (r=.87j while those provided with only a 
job title had low interrater agreement. A second means of iiii- 

proving reliability is interview structure. Schwab & Henemann 

' — -r - 

(1969) found highest interrater reliability when interviewers 

used a structured interview format. When a semi-structured 
format was used, reliability dropped (r=-43) ^ and when an 
unstructured format was used, the reliability coefficient 
dropped even further (r=.36) . Similar results were shown by 
Carlson, Schwab & Heneman (1970). 

The type of fating scale used has been shown to af j^ect in- 
terrater reliability. Maas (1965) conducted a study in which 
inter viewe^rs assessed job appl? cants for a particular job on 
two different rating scales. In the first study, interviewers 
rated appHcants on a traditional adjectival rating scale. The 
reliabilities Vere low for traits (r=. 35) , overall rating. 




meai 
that] 
1968; 

as five minute|*of traf|!p|^was | fou||f^^ 
effective in reducing perceptual errors. Shick (1973) likewise^ 
reported fewer errors in perception among raters exposed brief- 
ly to training. 

The best training pedagogy is still under investigation. 
Levine & Butler (1952) found only group discussion to be effec- 
tive in reducing "halo" error (a perceptual- error/ in which a 
wide variety of positive attributes are ascribed to the appli- 
cant on the basis of limited positive data) . According to 
Levine & Butler, th^ lecture method and experience had no 
effect in reducing the "halo effect." Brown (1968) also ex- 
araineS a variety of training methods and concluded that all 
methotte were effect! v^e in -reducing the halo error. Finally, 
Wexley, et al., (1973), and Latham, et al., (1975), fpund work- 
shops with exercises had the most dramatic effect in reducing 

rating errors. 
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Vaiiaity 

The validity of the interview has been assessed against 
such criteria as performance ratings, success in training, and 
job tenure. The ^results of most of these studies have been 
disappointing. In his review of the studies examining selee- . 
tion interviewing, Mayfield (1964) stated, "Although the relia- 
bilities of interview may be high in given situations, the val- 
idities obtained are usually of low magnitude. .This indicates 
that along with the present emphasis on reliability, there 
should be more investigation of just what it is that is being 
measured reliably in selection interviews." In Wagner's (1949) 
review of 106 studies, the mean validity coefficient reported 
for traits and characteristics was only .37 and the- validity of 
the overall ratings was .35. It must be kept in mind, however, 
that the number of studies which assessed validity was quite 
small and in those studies a number of- different criteria were ' 
used. Nonetheless, they serve to underscore the generally low 
validity of the interview as a iselection tool. This* state of 
affairs will probably remain the case as long as the inter- 
viewers are permitted to draw their own inferences unsystem- 
atically from the data they collect. 

Most studies that have examined validity of the selection 
interview have used job performance ratings as the criterion 
"(Raines & Rohrer, 1955; iaccaria, et al. , 1956; Woodworth, \ 
al., 1957; Trites, 1960; Campbell, et al., 1962; Huse, 1962). 
Other criteria have included the successful 



coinpietion of training (Flag, 1961; Trahkellr 1959) and job 




proach such as Rundquist's might well allow the interviewer to 
raise the validity of the interview by focusing on assessment 
of one area rather than many. The main area which appears to 
be most accurately'assessed by the interview is that of socia- 
bility or interpersonal relations; Otis, Campbell & Prien. 
(1962) concluded that the interview yielded valid predictions 
only on the personal delations dimension. Other evidence in 
•support of their conclusion is found in the studies of Holt 
(1958) and Loevinger (1959) . 

A number of other -factors also have a bearing on the 
validity achieved through the use of the interview. One cause 
for low validity may be errors in information processing, an . 
area which has been exposed to much scrutiny in the last 
decade. Springbett" (1958) and Bolster & Springbett (1961) 
found that the selection interview was used primarily to access 
negative information about the candidate. They concluded that 
though often unintended, the interviewer comes to weight 
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negative information too heavily^ (vis-a-vis its actual job 
relevance) when making an employment decision. In later 
research Hollman (1972) discovered that negative data are not 
more poten€ because they are given excessive inappropriate 
attention by d^ec^ but because positive inforraatiofi 

is overlooked or underutilized in the actual decision-making 
process. The later research would suggest that interviewers 
should be alerted or taught to attend to and use positive 
interview data more carefully in order to improve the validity 
of their interview decisions. 

The effect of training on validity has received some atten- 
tion. In research by Borman (1975) it was found that training 

programs designed to reduce "halo errox" did reduce halo but 

_ _ *- _ _ " * _ ' ■ 

left validity unchanged. There was some indication, however, 

that relative strengths and weaknesses of the applicants were 
recognized somewhat more accurately after training. 

Interview structure and the use of biographical information 
were examined by H^neman, et al., 19751 It was discovered that 
neither the degree of interview structure nor the use of bio- 
graphical data influenced validity * The authors speculate that 

_ _ 

individual judges may have been making decisions based on tfieir 

own unique stereotypes 'of the characteristics needed to ade- 
quately fill the job. This occurred because of the lack of ade- 
quate information about' desirable characteristics and behaviors 
of job incumbents. Indeed, as a study by Wiener & Schneiderman 
(1974) demonstrated, when information about the job vacancy was 



supplied to the. interviewer , more relevant and less irrelevant 
ihforihatioh was used in "the selection decision. 

Sydiaha's (1961) research suggested there is an "ideal" ap- 
.^plicant stereotype against which job applicants^ are judged. He 
• • / further felt this "ideal" stereotype was conunoh to all iriter- 
" viewers. However, in a later study, Hakel, Hollmann & Dunnette 
(1970) found that ideal stereotypes do appear to exist, but 
they are at' least partially unique to the interviewer, rather 
than being conunbn to all interviewers. 

Various authoris have expanded on the stereotype hypothesis 
to examine sex stereotypes which affect interviewer decisions 
(Cecil, et al . , 'l973 ? Shaw, 1972; Heneman, 1977; Cohen & 
Bunker, 1975). Mayer Se Bell (1975) examined sex stereotypes 
and found that different stereotypes of men and. women are re- 
sponsible for different- hiring decisions. Also, the authors 
revealed that the sex of the. interviewer plays a key role as* 
well. Female interviewers had more similar and less complex 

•stereotypes of men and women than did male interviewers* 

■> - 
Table 1 summarizes the reliability and validity of data 

obtained in the use of the selection interview and documented 
by the studies cited abbve. 

Issues 'for Users , " 

Perhaps the most crucial problem facing users of the inter- 
view as a selection device is its typically low validity. 
While studies have indicated that reliability can be raised 
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TABLE i 

Dimensions Measured by the Selection Interview 



Dimension 


Reliability 


Validity 


Traits and Characteristics 






Range 
Mean 


.15-. 98 
.56 


.17-". 71 . - 
.23 


intelligehae 

Range 
Mean 


.62-. 96 
.82 


.09-. 94 
.67 


Interpersonal f^eiations 






Rartge 
t^an 


.38-. 87 
.71 


.22-. 65 
.40 


Overall Ef f ectiverress. 






Range 
Mean . 


-.20-. 85 
.45 


.22-. 87 
.41 


> 
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through techniques such as interviewer training, structured 
interview format, use of job information and more sophisticated 
rating scales, the validity of the interview remains low under 
most conditions. 

The only ways which have been shown effective in increasing 
the interview's validity were restriction of data collected 
from the interview to interpersonal relations and the use of 
greater job information .ich is behaviorally based, Heneman, 
et al., (1975) stated the dilemma following a sophisticated 
study on interview validity. "A strong effort was made to cre- 
ate an interviewing process that would result in valid assess- 
ments in the structured interview condition." These efforts 
included (1) thorough job analysis, resulting in descriptions 
of basic job elements; (2) use of the job elements in the 
criterion measures; (3) use of interview rating forms requiring 
interviewers to make explicit predictions of performance oh the 
job elements; and (4) development of structured interview 
questions directly from descriptions of the job elements. In 
spite of . these efforts, interviewer validity remained low. It 
is thus necessary to asl^what more could reasonably be done in 
actual interview settings. ^ 

The solution to the dilemma lies in a two-pronged effort 
that would first minimize interviewer error and bias, thus 
raising reliability. Heightened reliability increases the 
possibility that validity could be raised. The second effort 
should be to obtain job competency analyses of positions to be 
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filled, tfiis information should be supplied to interviewers 
who would then interview candidates only for those positions 
where interpersonal relations were an essential skill to 
successful job performance. 

A related question is that of functional utility. Though 
the validity of the interview is low,- it may still be an ac- 
ceptable component of the selection • process, provided it makes 
a unique contribution to the data needed for a selection deci- 
sion. However, it appears that the data most easily and valid- 
ly assessed by the interview may also be assessed, perhaps more 
effectively, by other methods such as psychometric testing or 
thought samples. Some researchers (Huse, 1962; Flag, 1961; 
Ulrich & Trumbo, 1965; Grant & Bray, 1971; Wright, 1969; 
Schmitt, 1976) have already raised this issue, but relatively 
little work in this area has been conducted. 

Ah auxiliary question to that of relative utility is the 
issue of cost-effectiveness. Even if the interview can be 
shown to make a unique contribution to prediction of job suc- 
cess, tenure or training success, is the gain in predictive 
ability worth the cost involved? 

A third issue which must be addressed is the legal position 
of the. interview as a selection device. The interview has not 
come under attack to the same extent as other selection device^ 
under Title VII, such as psychometric tests. However, employers 
who use selection interviews may be called upon to give evidence 
not only for the validity of the^^ihstrument , but also for its 



fairness to minority groups and women. As mentioned earlier, 
it is clear that sex and race do impact interview decisions in 
quite complex ways. This evidence combined with the typical 
low validities and reliabilities associated with the ^^election 
interview are likely to make defense of the interview quite 
difficult. 

Psychometric Tests 

Testing as a tool for personnel selection has been in exis- 
tence for more than 50 years, receiving its first widespread 
use during World War I as a selection device for the U.S. 
military. The development of tests for selection expanded 
duriTig the following decades, reaching: a high degree of 
sophistication in the post-World War II periods Acceptance has 
not been as widespread as one might assume. Ward (1960) 
surveyed 1610 managers on the use of tests in their companies. 
Of these, 42% reported tests were usecJ for hourly employees 
while 53% said tests were used for exempt employees. Campbell, 
et ai^, (1970) estimated that 60-70% of the companies in the 
U.S. uce ability or aptitude tests, with many fewer- firms using 
personality tests. • 

There appears to be wider use of tests for selection deci- 
sions involving hiring persons from outside the firm. Ward 
(I960) found about 53% of companies surveyed used testing for 
external selection, while 36% used testing for internal selec- 
tion. This differential testing emphasis was supported by 
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Campbell, et al., (1970) who reported that approximately 9Q% of 
companies sampled used tests with external hires; only 40% 
tested for internal decisions. The rationale given for this 
difference was that testing is really helpful "only when little 
is known about the individual." 

Construction of Tests 

Duhnette (1966) has noted three basic methods for construc- 
tion of tests: (1) armchair theoretic, (2) factor analytic, and 
(3) empirical. 

1. The " atrmchaxr^ theoretic approach " involves devising . 
a set of materials or questions that will be used to elicit 
responses from persons, deciding what the responses mean, and 
then either confirming or disconf irming the actual behavior of 
those persons tested. This approach is used quite commonly, ^, 
but has little to reconunehd it since validities with job 
performance tend to be low. 

2. The factor-analytic approach involves describing and 
rating actual behaviors shown by persons. These descriptions 
are then correlated and factor analyzecJ in order to yield basic 
dimensions of behavior. Tests are then con.structed to measure 
these various factors. 

3. Finally, the empirical approach involves observation of 
differences in a particular behavior, rating or categorizing 
individuals according to the amount of the behavior they show. 



constructing stimuli which appear to be related to the behavior 
under study^ and then testing the stimuli to see which measures 
actually differentiate groups of persons having differing 
amounts of the behavior studied. . 

Of these three methods^ the most desirable is the empirical 
method because it is so strongly behaviorally based and tends 
to demonstrate the greatest validity. 

Administration and Content 

Tests differ substantially in their administration and con- 
tent areas. For example^ tests may be closely timed (speed 
test) or untimed (power test) ; they may involve "hands-on" 
manipulation (performance test)/ oral questions, or paper and 
pencil measures; they may have correct answers (objective 
tests) or no correct answers (subjective tests) . 

. The most important dimension on which tests differ is the - 
content they are designed to measure. Accordingly/ the hun- 
dreds of tests in use in selection decisions may be grouped 
under two major headings: ability and skill tests and person- 
ality tests. 

Ability and Skill Tests 

Ability and skill tests are theoretically different/ the 
ability test purporting to measure potential as opposed to 
level of acquired skill. In practice/ the main difference 
between the two tests is the purpose of the testing: the same 
test is frequently used to assess potential or actual skill, 
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Ability and skill tests differ according to the content they 
measure as well as the specificity of the dimensions tapped. 
General atbility tests usually measure general intellectual 
ability, while specific ability and skill tests measure particu- 
lar facets of intellectual, perceptual, psychomotor or other 
abilities. 

6eheral latellectu al ftbi l^ity— Tes£s 
Variety in Format 

There has been an evolution of general intellectual ability 
tests, beginning with the early, "spiral omnibus", tests. These 
tests assumed a general aptitude which would be measured, to some 
extent by all questions. The result of this type of measure 
was the single score derived from a number of different types 
of items. The spiral omnibus variety gave way to a different 
type of test which assumed a certain number of factors or dimen- 
sions comprising intelligence, rather than a single factor. In 
some cases, an overall score was given as well. Doppelt (1954) 
reported a growing trend for the factorial concept and a 
decline among single score tests. 

Usually, for personnel selection, intellectual ability 
tests are administered* to groups of persons rather than to one 
individual at a time. This allows the cost of testing to be 
kept substantially lower per applicant than would otherwise be 
the case. In addition, the time required for such tests is 
shorter than in other settings, again allowing a lowered cost 
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per person^ This need for ^group administered, short tests of 
intellectual ability has produced a wide variety of occupational 
tests designed to measure general intellectual ability. Notice- 
ably absent from the test used in industry is the Wechslef Adult 
Intelligence Scale, widely used in other settings, but which 
requires inc3ividual administration combined with a rather long 
completion time of approximately one hour* In contrast, meas- 
ures cbramohly used in industry average only 20-30 minutes for a 
complete administration and may be given to large groups if 
needed. They tend to be multiple choice speed tests which 
increase in difficulty. General intellectual ability tests are 
usually usec3 as a preliminary screening device to be followed 
by performance or skill tests. 

Instrument Focus 

JSom'e of the more popular tests designed to measure general 
intellectual ability are listed in Table 2, As seen in that 
*.^ble, there is a pronounced emphasis oh cognitive abilities 
with some attention to problem-solving ability. In particular, 
such dimensions measured as fluency with numbers and words, dis- 
covering relationships among words and general reasoning are 
tapped by most of the -tests coiijnonly used. However, dimensions 
such as flexibility, creativity, and problem diagnosis, also 
part of general intellectual level, are not measured by the 
tests* Guioh (1965) stated, ".,,the general intelligence tests 
have been conducted less than general because they do not 
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TABLE 2 

General Measures of Meiiectnal Ability CcDionly Used in Industry 



Rane 



Construction 



Reliability 



VaUdity 



Dimensions Measured 



Otisjelf-Aiiiaistering 
TfesEs of Mental 
Ability S-A . 



Spiral coDibus-single 
score obtained 

Paper fi pencil speed 
test 

30 minutes 

4 alternative forms 



Nui^ fluency General reasoning 
Xnowiedge i Spatial relation 



meaning of 
words 

Classifying 
verbal con- 
cepts 

Perceiving rela- 
tioniships among 
verbal concepts 



Ability to reason 

logically 
Perceiving events or 
concepts in 
logical order • 



I wonderlic 'Personnel Test 



I 



Spiral omnibus-single 
score obtained 

Paper & pencil speed 
test 

50 itoDS 

12 minutes 

9 alternative forms 



Test-ietest 
Range .82- 
.94 

Split-half 
.88-.94 



Number fluency Genez:al reasoning 
Knowledge s Sp&tial relation 



meaning of 
words 

viassifying 
verbal con- 
"Cepts 

Perceiving rela- 
tionships among 
verbal; concepts 



Ability to reason 

logically 
Perceiving ev^ts or 
concepts in 
logical order 



Adaptability Test 



Factor-analytic 
Multiple scores. ■ 

obtained 
Paper s pencil power 

test 
Spiral omnibus 
2 alternative forms \ 



.80 



.73-.79 



Number fluency General reasoning 
Knowledge fi Perceiving patterns, 

meaning of in geometric or; 

words verbal stimuli 

Classi^ing Arranging events or 

verbal con- .concepts xn 

cepts logical order 

Perceiving rela- • 

tionships among 

verbal concepts 



(Table 2, contiiaued) 



Nane 



Constzuction 



ReliabiUly 



Validity 



Dimensions Measured 



Ihozstone Test of Mental 
Alness 



Factor-analytic 
Paper s pencil speed 
■test 
126 items 
20 minutes 



Split-half 
.95 



Ni^r fluency General reasoning 
Knowledge s Perceiving pattens 
in gecmetric or 



meaning of 
words 
Classifying 
verbal con- 
cepts 



verbal stiodi 



Wesman Personnel Classifi- 
cation Test 



' Ghiselli Analysis of Rela- 



te 



tionships 
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Paper S pencil speed 
60 items 
28 minutes 



Mtemate 

form 

.73-.92 
^lit-half 

,82-.94 



Number fluency 

Discovering relationships among 
• verbal concepts 



Paper S pencil power 

test 
40 items 



Odd-even 
.82 



Con- 
current 
.22-.76 



Number fluency Discovering rela- 



Knotf ledge & 
meaning of 
words 

Classifying 
verbal con- 
cepts 



tionships among 
verbal concepts 
General reasoning 
Ability to reason 
logically 



measure the important intellectual powers involved in creative, 
thought, planning and judgments ' They do not even tap some of 
the low<Br level intellectual skills measured in tests of 
clerical o^ mechanical aptitude." 

Psychometric Properties 

Nonetheless, some" of the general intelligence tests have 
- proven to be usef^Sl additions 'to testing batteries for some 
jiDbs. Ghiselli (1973) examined all the validity studies . (1921- 
1971) using occupational samples. From these data, he completed 
the average validity coef f icients ,f or the criteria of training 
and job proficiency. Unfortunately, in computing these coef- 
ficients, concurrent and predictive validity studies using all 
general intellectual ability measures were combined, as were 
both well constructed and poorly designed studies. The validity 

figures attained, therefore, are likely to be underestimates of 

•> __ ____ ___ 

true validity. These validity coefficients which may be seen 

in Table 3 reveal some interesting trends. . (1) CSeaeral 

intelligence tests appear to be much better predictors of 

success in training than of job performance. This is to be. 

expected given the scholastic nature of most intelligence 

tests. (2) The utilitv of general intelligence tests is most 

apparen^^ih clerical and managerial occupations. Overall, 

.validities of general intelligence tests are low (between .15 

. r^nd .30) for all occupations. ' ' . ' 



rrasLB 3 

Validity of General Intelligence Tests* 



Occgpation Criterion 

Tr^iTiTTig Job Proficiency 

Managerial o c cupatioas .29 . .29 

Clerical occc^ations ' .30 

Sallys occupations — .19 

Protective *occt^tions (e.g., firemen, 

police) .65 .23 

Service oca^ations .42 .26 . , 

V^icle operators- <i21 .15 , 

Trades and crafts .41 .25 

Industrial workers .38 .^iO 



♦Adapted from Ghiselli (1973) 
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Specific Aptitude Tests 

Specific aptitude tests attempt to measure an applicant's 
potential oh a specified set of traits arid abilities. Of 
course^ in reality, true tests of potential do not exist, but 
are always influenced to some degree by* such factors as pre- 
vious exper i^hoe- ^nd-J^eaxriXng-or mot i vatiori . . Specific aptitude 
tests may be combined to form a multiaptitude battery with each 
subtest designed to measure different dimensions of. potential 
for a given job. 

Instrument Focus 



Specific aptitude and skirll tests exist for a variety of 
dimensions, but in personnel selection the most prominent areas 
are: 

• Specific intellectual abilities including the ability to 
deal with verbal and numerical materials arid geometric 
forms. Perceptual abilities and memory are often impor- 
tant aspects as well* Specific intellectual ability 
tests have been most often used to assess clerical 
aptitude. . . . 

• Mechanical aptitude measuririg spatial orieritatioris or - 
visualization. These tests, may be paper and pericil 
measures or require actual object manipulation. These 
skills are most often measured for trades and crafts- 
people, such as electrical or structural workers, or 
some machine operators. 
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• Cr^tivitv and indgmeht which explores some of the 
higher intellectual processes omitted ii general in- 
telligence tests-. The traits tapped by creativity and 
judgment tests include the ability to make inferences, 
recognize assumptions, deduce logical conclusions, and 
evaluate arguments. . They have been most used in man- : 
agerial and scientific professions. 

• Sensor^and^ perceptual capabilities including such 
aspects' as vision acuity, depth perception, and audi- 

" tory acuity. Such traits are especially important to 
clerical workers, inspection workers, vehicle opera- 
tors, machine operators, some laborers, and mechanic and 
skilled tradespeople. , 

• Psychomotor- abil i ty including dexterity, eye-hand co- 
ordination and object manipulation. These abilities are 
most needed in work requiring speed and object manipula- 
tion, such as assembly of small components or electrical 
wiring. 

Table 4 lists some, of . the more common tests in each of 
these categories as well as the <? .mensions of facets of 

behavior measured by each. 

✓ 

Psychometric Properties 

Specific Aptitude Tes£s . The average validities of specific 
ability tests for eight job families were presented by Ghiselli 
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Minnesota 



General Cl 



Short Etaiploym( 



I 

00 



^ Minnesota Pap< 
Board Test 



Survey of Spac 
Ability 



Christensen-Gi: 
Tests 

Owens* Qreati'^ 
Machine Desi 



(Table 4, continued) 



Name 


Construction 


Reliability 


Validity 


bimenH 


Watson-Gleser Critical 
Thinking Appraisal 


Paper & pencil power 
test 

8 tests - 230 items 
No time limit 


Split-half 
.50-. 84 




Inference H 
Recognition 
Deduction H 
InterpretaS 
ptf ttTtistion ^1 


MacQdarrie Test for 


Paper & pencil 


.70-. 89 




Spatial 


Me^anical Ability 


7 subtests 






Controlled M 
Visual inspel 


O'Connor Finger & Tweezer 
Dexterity Tests 


Object manipulation 
speed test 


Test-retest 
.89-. 93 




Manual dextei 


^ Purdue Pegbocuxl 
* 

ID 


Object manipulation 


Split-half 
.82-. 91 


.07-. 76 


Manual dextex 


1 _ 

Minnesota Rate of Mani- 
pulation Test 


Object manipulation 






Manual dextex 
Finger dextex 
Wrist-finger 
Positioning 


•15 
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(i973) . These may be seen in Table 5. As with ^ener^ Xoii^^ 
gence tests, it appe.ar^--^s--tf-^TaTni^^ is more accu- 

rately predicted by aptitude tests .than -job proficiency, 
jajifoii^fetinatery/'^^^ he did not report validities 

of measures of creativity with training or job performance. 
However, what data there are tend to be concurrent studies 
which show reasonably good discrimination among creative and 
non--creative engineers and programmers (dwens/ et al., 1957; 
Lahgmuir & Kendall/ 1961; McNamara & Hughes, 1961). 

Multlaptitude Test^^tteri^ . Test batteries which assess 
aptitude in a number of areas have increased in popularity in 
recent years. The advantages of such batteries are greater 
efficiency in the use of testing time, and a greater amount of 
data obtained about the applicant. Such batteries may be used 
to assess the wide variety of skills which may be needed to 
successfully perform the various facets and tasks of a 
particular job. The use of multiaptitude test batteries has - 
been most prevalent in the military though recently private 
industry has begun to Accept and use ^ome batteries. Table 6 
summarizes data from four major multiaptitude tests in current 
use. 

Personality Tests 

!gari^ety ih^ormat ^ . 

Personality tests are designed to measure the emotional/ 
interpersonal/ attitudinal and motivational facets of ah appli- 
cant. Per^^ measures originated in guidance centers and 



Validity of l^ecif ic Aptitude tests* 



jQcc u p a t i oiT ^ Ability Criterion 

•~ " training - Job Proficiency 

Managerial occupations Intellectual. .30 .^7 

Spatial/Mechanical .28 .22 

Perceptual .23 .25. 

Psychomotor .02 -j" il4 

Clerical occupations ' Intellectual .47 .28 

Spatial/taechemical .34 .17 

Perceptual -40 ' .29 

Psy^cmotor .14 .16 

Sales occupations Intellectual — .19 

^atialy^echanical — .18 

Perceptual .04 

Psychomotor — .12 



Protective occixpations Intellectual .42 .22 

^atial/teechanical .35 . .18 

PerceptuetL .30 ^ .21 

Psychomotor — » .14 

Service occupations Intellectual 4 42 .27 

^patialAechanical .31 .13 

Perceptual .25 .10 

Psychomotor .21 .15 

Vc^cle operators fetellectual .18 .16 

^tiid/taechanical .31 .20 

Perceptual .09 .17 

Psychomotor .31 .25 

trades & crafts . Intellectual .41 .25 

^atial/medJanical .41 .23 

Perceptual .35 .24 

- Psychomotor .20 .19 

• . " ]^ ' 

Industrial workers intellectual .38 .2.0 

Spitial/Secbanical .40 .20 

Perceptual .20 .20 

Psychomotor .28 .22 



♦ Adapted from Ghiselli (1973) 
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TABLE 6 $ 
Hultiaptltude Teist Batteries 



Name 



Cotistsniction 



Pellabilltv 



Validity 



Dime 



Differential Aptitude 
Tests 



Paper & pencil power 

test 
4 hours 

2 Alternative fonnis 



Alternate 

form 
.73-. 94 
Split- level 
.96- .99 



Predictive 
-•23 +,23 



Verbal re 
Numerical 

ability 
Abstract 

reasonini 



Aptitude classi- 
fication Tests 



Paper & pencil speed 

& power tests 
19 tests 
10^ hours 



to 



Split-half 
: •65-,86 
Aitemate 

form .55- 

• 85 



Predictive 
•04-. 65 



f 



Verbal f lu^ 
Numerical 

ability 

Judgment & com 

prehension 
Inspection 
Coding 
Memory 
Precision 
Scale reading 



General Aptitude T^st 
Battery 



Paper & pencil 
12 tests 



Verbal fluency 

Numerical 
ability 

Finger Dex- 
terity 

Manual Dex- 
terity 



Qnployee Aptitude Survey 
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Paper & pencil 
iO tests 
55 minutes 
2 Alternative forms 



Aitemate 
fbm .60- 
.70 

Test-retest 
.76-. 84 



Verbal corn- 



Numerical 
ability 

Verbal 
reasoning. 

Numerical 
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I relations 

Leal 
honing 
pal speed 
3curacy 



repro- 
^lon 
ileal 
masoning 
lanlcal 
xnponent 
Le reading 
cdinatioh 



sral ii- 
diligence 
bial relations 
a perception 
deal pereep-. 
ton 

cdination 
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mental hospitals, then moved into industrial persohh^ selec- 
tion on the assumption that personality is an important deter- 
minant of job performance, job tenure and absenteeism, and gen- 
eral quality of life. The number of personality tests is how 
in the thousands, making test selection frequently c3ifficult. ' 

There are two basic forms of. personality measures: the 
self-report inventory and projective techniques. The self-. 
. report inventory requires the applicant t o indica te how, descrip- 
tive statements or adjectives relate to himself or herself. 
The primary difficulty inherent in this approach is- the oppor- ' 
tuhity for "faking" or giving only socially* desirable answers. 
\ Particularly in personnel selection, there may be a strong 

desire on the part of the applicant to "look as good as pos- 
sible" in order to be selected for the position. 



\ Several procedures have been tried to decrease or eliminate 

\ — 

_ V _ _ _ __ _ . ■ _ _ _ _ _ _ . _ _ _ _ 

the effect of the social desirability bias in the use of self- 
report measures.. One solution is the use of "forced-choice" 
measures in which the applicant chooses between two or three 
possible descriptors, all keyed to have the same desirability. 
There is some evidence to. suggest this minimizes the effect of 
the "faking", but still does not eliminate it as a. source of 
bias (Wiggins, 1966) . A second way to counter the problem 
of "^faking" is the construction of special keys which assess 
the degree to which responses appear to be heavily influenced * 
by wanting to appear , in a more favorable light. These keys 
also appear to reduce the effect of the bias, but do not 
eliminate it (Gofer, et.al., 1949).. ' . ' 



other difficulties which beset the use of . self-report in- 
ventories is that of response sets of styles. Inventories lend 
themselves to such biasing response patterns as abguiescence 
(the " tendency of the applicant to agree with statements) or 
deviance (the tendency to give unusual or uncommon resp5ni§§)'; 

A' second form of personality measure is that of* projective 
tests. The primary characteristic of projective technique is 
the use of ambiguous or unstructured stimuli. It is assumed 
that by using ambiguous stimuli r the applicant is free to pro- 
ject his or her own desires ^ emotions^ needs onto the stimuli r 
and structure the situation according to , fundamental aspects of 
personal psychological functioning. 

Projective techniques are disguised in that the applicant 
■seldom is aware of the interpretation thatwill be made of par- 
ticular responses. Thus^ faking and response sets are not a 
problem with this type of technique. Unfortunately r most 
projective techniques, with the exception of those utilized to 
collect thought samples, lack. standardization of administration^ 

and scoring. This makes results obtained suspect, due to lower 

_ J * _ ______ _ _. / _ 

inter- and intrarater reliabilities^ Additionally, many of 

the project ivTlneasures use subjective rather than objective 

scoring techniques. Normative-^ta, as well, are frequently 

not available, especially on industrial samples. 
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Interest Measures 

Interest measures were devised originally for clinical use 
in guidance counseling rather than employee selection. These 
measures are typically self-report instruments requiring a . 
rating of a' particular activity, or a preference among several 
activities. " \ 

The two most commonly uised interest measures are the Strong-^ 
Campbell Interest Inventory (ah earlier version being the Strong 
-Vocational Interest Blank), and the Kuder Preference Record. 
In the sen, applicant responses are compared with responses of 
people in various jobs. ' These interest similarities and dis- 
similarities are charted for a number of occupations allowing 
the applicant to gauge how closely his or her interests match 
interests of people in those occupations. The Kuder Preference 
Record requires the applicant to indicate preferred activities 

in a forced-choice format. The results are given in strength 

* ___ 
of interest traits, such as mechanical, scientific, a^istic, 

etc.., rather than in terms of occupations. In both interest 

measures there are verification scores to detect faking or 

carelessness. Other interest measures exist, but are -seldom - '/ 

•■ » ' 

used in industrial settings. 




I ns t rument, Focus 

Personality tests also vary according to the content and 
purpose of the test.. The, two dominant forms of cont^ent of per- 
sonal'ity tests' aife {1} Motives and Traits, and (2) Interests. 

Measur^es of Mbtives ^nd Traits ' 
A sampling of the -many motivational and trait measures used 
for selection purposes" are shown in Table 6. No one test seems 

to be used most frequently^r though projective tests are used 

less frequently than self-rsport measures, probably due to the 
difficult scoring procedures typical "of these measures and t3ie 
low interfater reliabilities'. , Most of the measures 6£ motives,, 
and traits tap rather ambiguou*s .personality dimensions such as 
dominance, extroversion, stability or masculinity, which may or 
may'not be'reflected in behavior on the job. Cronbach (1960) 
criticized trait measures by challenging the assumptions oh 
which the tests are based. The assumptions according -to him. 
are: . " 

• Personalities possess considerable consistency; a per- 
son shows the same habitual reactions over a wide range 
of similar situations.: 

• For any habit we can: find, among people, there is a 
variation /of degrees or amounts of this 'behavior . ^ 

• Personalities have some stability,' since the person 
earning a certain score this year usually has a somewhat 

. similar score next year. 



Each of these assumptions is highly questionable. It seems 
unrealistic to assume persons will react Similarly in diverse 
situations. Rather, traits should be measured in "terms of 
behavioral tendencies with a defined class of stimulus situa- 
•zions" (Guion, 1965) that most approach the work situation. 
Secondly, the assumption that variations of traits are great 
enough among average people to allow such trails to be used as 
predictors is questionable, , Such measures are likely to pick 
up the highly unusual cases ^ but-J.ii if act> "ther ma jor ity of per- 
sons likely to be tested for positions in industry are unlikely 
to be situated at the extremes on a measure of a particular 
trait. Finally, personalities show some change over time whiqh 
is reflected in low test-retest reliabilities on personality 
trait measures (McClelland, 1980). ,^ 

Psychometric Properties , Reliabilities and validities for 
particular motive and trait measures may--be Seen, in Table 7 . 
Further, the validities of these tests for various occupational 
groups may be seen in Table 8*. The results of this table show 
good validity of these measures for predicting training success 
of managers and moderate validity for" 30b performance of sales 
personnel. For other groups, however, validities are quite low 

lieliabilities of measures of motives and traits have long 
been a problem. Not only have these tests shown lower 
reliability over time, but with projective techniques in 
particular, interrater reliability is frequently low (Ahastasi, 
1968) . 
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Personality Tests Camnonly Used in Industiy 



Naoe 



Minnesota Multiphasic 
Personality Inventory 



Constructioi 



Onpirical 
Self Report 
Paper s pencil 



SfiliabUity 



Validity 



Con- 
current 
-.33-+.37 



Dinenslons Measured 



Psychological Dysfunction 
Masculinity-Femininity 
Social Introversion 



California Psychological 
Inventory 



Empirical 
Self Report 
Paper S pencil 



Con- 
current 
.44 



Psychological traits S needs 
Intellectual £ interest patterns 



Guilford-Zinmennan Tem- 
perament Survey 



Factor-analytic 
Self-report 
Paper s pencil 



.S2-,92 



eon-' 

current 

-.i7-+.28 



Psychological traits 



Thurstone Temperament 
Schedule 



J 



Cattell 16 PF Question- 
naire 



Factor-analytic 
Self-report 
Paper s pencil 



Psychological traits' 



Self-r^rt . 
Paper fi pencil 



Psychological traits 



k Thematic Apperception 
^ Test 



Rotter Incomplete Sen- 
tences Blank 



Projective 
20 pictures 



Psychological needs & traits 



Projective 
40 items 
Paper s pencil 



-.73H^.70 



Psychological adjustment 



Miner Motivation to Manage Scale 



Projective 



Straig-ean^H interest 
Inventory 



Empirical 
Paper fi pencil 
Self-report 
280 items 



.75-.84 



.24^.32 



Congruence of interest 
Patterns with job incuiibant 



Ktider Preference Record 
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Predic- 
tive -.42- 
+.44 
Con- 
current 
.41 



Interest in different activities 
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TABLE 8 

Validity of Personality Tests* 



bcoapation 

Managerial occx^ations 
Clerical occi^tions 
Sales occupations 
Protective occupations 
Service occupations 
Vehicle operators 
Trades & crafts 
Industrial workers 



Gri4;erion * 



Training 
• 53 
.17 

-.11 



.16 



Job Proficiency 
.22 
.22 
.32 
• 21 

.21 

.26 
.24 
.26 



♦Adapted from Ghiselli (1973) 



Psych ometrlc^i^a per tie s . As with motive and tirait iheasuresv 
predictive validities of interest measures for performance of 
various occupational groups have been shown to be generally 
low. However, the validity of interest measures to predict 
success in training for managers is quite high. One way which 
has been shown to- be a particularly effective means of boosting 
the validity of interest measures is through the use of 
empirically derived scoring keys ABoyd, 1961; Knauft, 1951; 
Tiffin & Phelan, 1953)\ 

In contrast to motive and trait measures, interest tests 
show remarkably good reliability over time". In a study of the 
SVIB, Strong (1951) conducted follow*up testing with time 
intervals of 5 to 22 years. He found reliability of interests 
were .84 after five years, .82 after 10 years, and .75 after a 
22-year . inter val. - 

Issues For Users 

The use of tests in industrial selection appears to have 
come under a great deal of attack in recent years. Ebel (1977) 
states that while the -^attackers are varied, the target of attack 
is always the same: the alleged lack of validity of the tests. 
As we have seen earlier, validities of personality tests and 
general intelligence tests on job performance tend to be es:c- 
cially low for all occupational subgroups. Reasons for this 
lack of validity have been postulated by a number of researchers 
and include poor test construction, poor test selection, and 
differential validity. 
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Test Construction 
. Dxinnette (1966) has urged a move away from "armchair the- 
orizing" as the basis of test construction to mpre Sophisti- 
cated , behaviorally^based approaches. Yet, a number of tests 
are constructed on the |)asis of "pet theories" without adequate 
attention being paid to the actual behavicis indicative of 
success on the job. bunnette states, "Most existing behavioral 
theories have at best doubtful validity, and it is unlikely 
that any test developer is so' omniscient that he or she can 
accurately intuit what a person's responses to a set of stimuli 
may mean in terms of later observed behavior." 

In addition, most tests have been constructed to be adapted 
to the widest variety of industrial situations. .This feature 
lowers, the ability of a test to predict the unique behaviors 
necessary for success in a ^specific job. Instead, tests should 
be made much more "situatiohally specific", i.e./ constructed 
on the basis, of behaviors which differentiate successful from 
unsuccessful performance for a particular job. 

Test Selection ^ 

Some rationale is always needed for the inclusion of .par-- 
ticuiar tests into a selection battery. Unfortunately, accord- 
ing to Bray & Moses (1972), such rationale is frequently based 
on test availability or on intuition. A study by Parry (1968) 
illiastrates this point. She asked 10 industrial psychologists 
to estimate the validities of a number of tests widely used in 

-1.51- 



personnel selection^ Her results showed "only- one person was 
able to' achieve an accurate estimate of validity, with the 
other psychologists showing a marked tendency to overestimate 
• the validities* This overestimate would encourage the use of 
widely known tests without assessing the utility of the 
instrument far the particular job. 

As shown earlier, tests predict better, for some, groups of 
individuals than- others. Thus, inclusion^of a personality test 
in a managerial battery to predict success in a management 
training program may be a useful addition* However, use of the 
..same test to select police recruits who would successfully com- 
' plete training at the Police Academy would likely be inappro- . 

priate. Therefore, tests must be examined for their utility in 
predicting a certain outcome for a given occupational group. 

Differential Validity ; 
" ■ Art additional factor > in the attack on test validities is 

the possiblity that tests may be more valid for some groups 
• than others, i.e., differentially valid. A number of studies 
have claimed such differential validity exists (Bass & Turner, 
1973; Kirkpatrick, et al.,;1968; Tenopyr 1967; Bartlett & 
O'Leary, 1969; Boehm, 1972) while other studies have claimed 
differential validity is merely an artifact of poor criterion 
design (Campbell, 1943; Crooks, 1972) or other methodological 
problems (Schmidt, et al., 1974). A later section of this 
chapter cJeals with this issue in greater depth. 
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. In light of the questionable validity of many tests , as 
well as the recent attention testing has received in court 
cases, many companies are re-evaluating the utility of 
maintaining a testing program as part of -personnel selection i. 
Where tests are being retained, they appear to be much more 
behaviorally focused with less attention being given to person 
ability or general intelligence measures. 

Other Selection Devices 

This section is devoted to a discussion of the use of 
application blanks, resumes, r ecommenda t ions ,^ work sample s and . 
simulations, thought samples and detectors of deception. These 
devices are dealt with as a group due "to their less frequent 
use' and the fact that^ relatively few studies exist th^t attempt 
to describe the reliability and validity of these data sources. 

Application Blanks 

Many organizations judge a job candidate's potential on the 
basis of background information about that candidate. They 
reportedly consider- biographical data, educational background, 
work experience and/or performance on a previous job. One of 
the "easiest" ways to access some of this information is through 
the use of an application blank. It is easy because the blank 
can be designed to include questions on exactly those background 



areas in which an employer is interested. The form.. 



which can 
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be reproduced cheaply , provides a standard set. of questions that 
can be given to all candidates. Using an application blank, the 
prospective employer can gather comparable data from a number 
of candidates without investing a significant amount of time in 
gathering that information such as would be required in prelimi- 
nary interviews; a receptionist or secretary can simply ask all 
applicants to fill one out. Eater , candidates can be compared 
and those most qualified on the basis of the self-report appli- 
cation forms may then be interviewed or hired • ^-^ 

Variety in Fdritat 
• Most application forms look somewhat alike, though they may 
vary significantly in the amount of thought that was applied to 
designing them and the amount of information that they provide* 
Most include spaces for indicating name/ address, age, weight/ 
schools attended and dates of attendance, period of mil:»tary 
sefvi::e, previous jobs held, and names of personal references. 
More detailed applications may probe these areas for additional 
job-related information such as college grades, area of academic 
concentration, favorite subjects, type-of military discharge, 
responsibilities in previous jobs, the number of people super^ 
vised and reasons for leaving a job. Stii:^ other application 
blanks ask specifically if the candidate has held a job similar 
to the one presently applied for, -or ask for the candidate's 
career plans* The more general forms ^f the application blank 
rare -less expensive, ana are usually bought in large quantities 
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froin a business form supplier who imprints the organization's 
name at the top. The custom-made application blank which asks 
numerous detailed questions gives much more specific job 
related information at the greater cost incurred by analyzing 
the specific job to generate questions related to important 
areas of job functioning. The general form can be used for a 
variety of . jobs? the specific form is only appropriate for jobs 
that require a particular kind of background and preparation. 
, While the majority of organisations have all applicants 
. complete application blanks r many of those organizations would 
be surprised to hear the application blank called a selection 
device. Most firms use a general form for entry into, all areas 
of the organization. While such forms help the organization to 
gather data, these data are used in ah idiosyncratic manner ^ 
unspecified weight being given to that information' which may be 
combined with interview test or reference data. A small minor-- 
ity of organizations use "weighted"^ application blanks / which 
are empirically keyed to and validated against job performance. 
Weighted application blanks are also used as selection devices^ 
either by themselves or in combination with other data. 

Instrument Focus 

What information do application blanks provide to an 

employer? At the most basic levels the devices can give basic 

... /....._.. ... _.. 

indications of literacy^ spelling^ grammar and usage^ as well as 

neatness and the ability to follow simple .directions^ and they 
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provide verifiable age, education, military service and employ- 
ment histories* If accuritely recorded, these data provide an 
uncomplicated prof ile of applicant skills and job related 
experience. Que^*:iohs about one's reasons for leaving a job, 
or'^one's career goals, oh the other hahd,^ may provide a, crude 
reading on the motivation of the applicant. This information 
may be .helpful in deciding if an applicant would "fit in" .on 
the new job. However, as the^ self-r<:5port information on the 
application becomes less objective, and therefore less verifi- 
able, it also can be less detectably faked. All self-report 
knowledge, skills and abilities should probably be. verified, 
using the application blank only as a preliminary- screen to 
identify potential employees rather than for making hiring 
decisions about jobs that require those knowledges, skills and 
abilities.^ 

Psycfeometric Properties 

Reliability . The reliability of an application^ blank, 
generally speaking, is not at issue. One's education, job- 
history, and the opinions of one's references would not be 
expected to change very much over short intervals, to vary with 
ch^rTges in the format of an application blank, or to contradic€ 
other pieces of information on the same form. There is, none- 
theless, the potential problem of candidates faking information 
in order to present the image they think will get them jobs. 
Early researchers found self-report data on work history, and 



related information to be highly reliable, correlating approxi- 
mately .94 with verified data (Keating, et al., 1950; "Mosel & 
Cozen, 1952). More recent research suggests that applicant^ 
information may disagree with verified' information as much as 
57% of th? time (Goldstein/ 1971). However, among incumbents^ i 
who heed hot sell themselves for the job (Cascio/ 1975) r and j 
among applicants led to believe that a faking scale is included 
in\he application blank (Schrader & Osburn, 1977) , verified ,. 
responses indicate that application blank data are highly 
reliable. . 

Validity ^ - The validity of biographical items on application 
blanks, as compared to personality measures and other predic-vr^ 
tors, has been §ood for the prediction of job performance * I; 
(Asher, 1972); biographical items have outper foriSed -iiitelii-;^- ^^^^^ 
gence, aptitude, ptsychomotor , perception and p.ersohaiit^^ 
as predictors of job proficiency, / Ifc-^s ^^^^^^^^ T^^^^fv-'- 
however, that it is a rigorously. jaefiSed suBse^ ofitK^-iun^^ 
of all types of biographical data which h>s i&ared in 
such comparisons • Asher. r'ep(^rted.-.and compared: only uses of 
"hard" biographical items, items: which have been..<;ross- 
validated, and which have been combined as a predictor based on 

a set of biographical questions rather than used as single 

_^ ' _ _ __ 

items. Of 11 studies and— 31 validity coefficients meeting this 

set of criteria, all but 3% achieved' validities greater than 

•30, and 35% of the reported validity coefficients were in 



excess of .60. Many employers, however, use data from appli- 
cation blanks which may include "soft" unverifiable items which 
can be faked easily. Also, many users simply apply a scoring 
key to an application blank and use the key for selection 
without ever cross-validating it on another applicant sample to 
identify the dependence of predictiveness on chance sample 
variability. Still^cther users attend to individual items on 
.".the application blank, ultimately using one or a few of them as 
predictors instead of using the combined biographical profile 
which will',^by definition, be able to explain "more of the vari- 
ance in job performance. Asher drew his conclusion about the 
utility of biographical items from -a sample of methodologically 
sound uses. Users of biographical data (Thayer, 1977; Roach, 
1971) reporting validities between , .30 and .40' have also argued 
forcefully for use of. application blanks which are continually 
cross 'validated aitd updated to match the job-relatedhess of 
background items for new applicant populations. 

igor is reguired^'to maintain validities even at a moderate 
level '(r=-p35) . Economic and organizational climate, as well as 
changing personnel practices, have a continual influence oh tHe 
predictive validity of information gathered on an application 
blank, ft it is acknowledged ' that none of these factors is - 
static, then the good predictive validity of biographical data 
(e.g.-, personal', education and job history information) from 
application blanks is something that must constantly moni- . 
tor ed,; and changes should, be made when necessary to preserve 
validity. . 



Gti'iity ahd-validity of- .thfe. application blank for gathering , 

** * . 

predictive biogfapfiical data has been demonstrated for a variety 
of jobs types and job environments. Keyed application blanks 
have been used in the insurance industry since 1922, yielding 
cross-vaiidated validity coefficients around ,45 (Roach, 1571) • 
Typical of the validities obtained for different jobs are .SB 
referenced to performance in training and on the job for Navy 
divers (Helmreich, , et al.,^1973), .36 for men in the Israeli 
Army (Nevo, 1976) , and .6'^ referenced to quality of performance 
among dance, theater, music, and- visual arts students (James, 
et al. , 1974) . 

One study of the application blank examined both its reli- 
ability' and its validity for prediction of job performance. In 
a crosscultural study of American and Western European salesmen, 
\a biographical- inventory demonstratec^ good; median internal con- 
.sistency, 'or reliabilities, of *75. and a median validity coef- 
ficient of .42 (Hinrichs, et al. , 1976). The 48-point range of 
validities, however, reinforced the author's point that sample 
variations and variations in personnel practices necessitate 
validation of the keyed applications from each company and work 
group despibi perceived similarity of job titles and job 
descriptions between groups. Similarly ,, the variation in reli- 
ability estimates between groups (a 29 point spread) raises 

*j - - - - _. 

questions about whether experienced job applicants are less 

> _ 

reliable in their responses, and. whether that Ibwer reliability 
suggests an effort at self-presentation which decreases relia- 
bility and thence, validity. ^ * 

ss 
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Issues For Users 

At this point it should be clear that beyond the basic con- 
cerns of reliability and validity, there are several other seri 
ous issues to be considered when using an application blank as 
a selection tool* The issue of fakeability is a serious one; 
faked responses may reduce both reliability and validity of the 
method. Cross-validation is also important: a validity coef- 
ficient for a sample oh which the device is keyed will often be 
reduced to insignificance when the key is applied to a sample 
which doesn't have the unique pattern of variance the original 
sample had, A third issue is , one that was passed . 7er earlier; 
biodata validity may be different for different applicant sub- 
groups divided on the basis of age, sex, race, - education, and 
other background variables. One large trade organization has 
developed close to 50 different keys to scoring its biodata 
blank due to changes in applicant populations over time and to 
differences among subsamples of the population (Tliayer, 1977)- 
!Sfc^ A fourth issue concerns the nature of published studies. It 
has been argued (Schwab & Oliver, 1974) that few studies have 
been published oh the validity of applications' biographical 
data because they are seldom valid predictors of performance. 
Studies finding no validity, or lost validity upon cross- 
validation, tend hot to be published, but this is difficult to 
verify. 

-1.60- 
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Lastly, a most serious concern for practitioners today is 
the legal def ensibility of a selection technique as "job 
related". It has been suggested that data from the application 
blank be weighted according to their job-relatedness (Pace and 
Schoenfeldt, 1977) in order for the application to be a job 
Heated selection device. This complicates what was originally 
ah empirically simple method. Nevertheless, emp>loyers who are 
conimitted to using biographical data of the types included in 
an application blank may be able to meet legal requirements and 
develop a valid scoring key in this way (Cascio, 1976). 

Resumes 

Many organizations screen, or.even select, applicants on 
the basis of resumes they submit. A resume is similar to an 
application blank in terms of the information it supplies to 
the potential employer. .The resume, however, presents a less 
objective; less standardized self-portrait by a job candidate. 
Like the.-,s^lication blank it suffers from the typical weak- 
nesses of self-report measures. Applicants present themselves 
ai they wish to be seen, including only what they want and 
elaborating on information expected to cast them in a positive 
light. A resume often may Wt include specific job related 

information, such as reasons; for leaving the previous job and 

•. ■ \ 

school grades, that the employer iight desire. 

A resume is prepared by the applicant and given to the pro- 

L _ __ - 

spective employer. It typically includes a personal section, 

including information on age, marital status, and number of 



dependents y memberships and offices held in social and profes- 
six^l organizations • Ah education section will list the 
scfioo^ attended, years in attendance, and degrees -obtained , 
and may also include grade-point average, academic honors, con- 
centration of study or areas of special interest* A section on 
job history may simply list organizations and job titles or may 
be expanded to give varying amounts of detail on responsibili- 
ties, descriptions of projects and salary history, and profes- 
sional applicants may also list job-related publications or 
public presentations. Clearly, considerable variety in content 
and specificity may exist among even a small sample of resumes 
for the same job. 

Resumes do provide the employer with biographical informa- 
tion about the job candidate which includes some indication of 
personal 4^£e rests, information ^oh the candidate's education, 
and related job experience* Any self-report information on < 
knowledge, skills and abilities, however, does not necessarily 
constitute definitive evidence of them* These self-reports may. 
be more or less accurate depending upon the distortions (inten- 
tional or unintentional) inevitable in self -presentation. To 
the best of our knowledge, ho evidence of the reliability or 
validity of resumes exists.. However, all those who have writ- 
ten resumes recognize that they are written to specific audi-- 
ehces, for specif ic* jobs and are often rewritten for new audi- 
ences and opportunities. This would tend to reduce slightly 
both reliability and validity of the resume as a data source. 
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In addition to applicant biases, there are situational and 
personal biases introduced by the decision maker which will 
affect the use of resume data. For example, it has been shown 
that a resume will be evaluated differently depending upon the 
quality of the preceding, resume which serves as ah unwitting 
standard for. comparisons (Hakel, et al., 1970). While these 
contrast effects on resume evaluation are real, they account 
for a very small portion of the variance (about two percent) in 
the ultimate interviewer decision* If a candidate is only 
mediocre in qualifications on a resume, however, being preceded 
by a terrible candidate may make him or her look good by 
contrast, and result in an offer of a follow-up interview. 
Resume evaluations also tend to be more positive for attractive 
and qualified male candidates (Diploye, et al., 1975, 1977). 
Content areas, scholastic history, interest, and experience are 
all important inputs into resume ratings > which tend to be a 

function of the importance of the particular content area for 

. _ . _ _ . _ _ . _. . _■ 

job performance and the favorability of the information (Hakel, 

et al*, 1970). A systematic review of the biases involved in 
resume evaluation may be found in Arvey (1979). In sum, there 
are both job-related and unrelated influences which affect the 
validity of employment decisions baised ostensibly on resumes. 

In conclusion, the resume is widely used b^ecause of the 
potential wealth of information it can provide about a candi- 
date's personal, educational and job histories* It is, how- 
ever, a biased form of data, but one which, is used for a first 



-1.63^^ 



cut at whom to invite for a later interview. If used as a 
prescreen for standard selection procedures, the basis for 
choosing resumes should be demonstrably job-related or else it 
may be vulnerable to legal contest. Those who resumes as 
the basis of . selection, or as a prescreen should also be aware 
of non job-related biases which may affect and invalidate those 
judgments. 

Recommehdatibris 

Employers or personnel directors may consider recommenda- 
tions from acquaintances, other employers or colleagues, or 
individuals suggested by the applicant, when making decisions 
about a job candidate. As with application blanks and resumes, 
recommendations may be used directly to make hiring decisions, 
but are more likely to be used as the criterion for inviting a 
candidate to an interview. While some employers say they value 
a phone call to a past employer or the verbal opinion of a Col- 
league, only recommendations submitted in writing may be studie* 
in any systematic way. Therefore, the present discussion is. re 
stricted to written recommendations requested by the employer. 

A written recommendation may take a variety of forms. The 
'prospective employer may simply request a certain number of 
reference letters of the applicants or a form letter may be 
sent to persons named by the candidate. This letter may vary 
in structure ranging from a space for candidate and reference 
names, some simple directions and perhaps guiding questions 



specifying length and content of response, followed by -a blank 
space for writing, to a detailed questionnaire including free 
response, multiple choice, ranking, and forced-choice questions 
tapping information pertaining to work habits, personality, 
employment history, and whether the reference person would hire 
the candidate. Both the more and less structured forms have . 
the advantage of letting the referring person communicate those 
things about the candidate that he or she knows uniquely and 
which can't be obtained from an application blank or interview. 
At the same time, multiple unstructured references for candi- 
dates for a single job may vary in content, quality and speci- 
ficity. Comparison of unequal data and reconciliation of con- 
treidictiohs is difficult under these circumstances. Oh the 
other hand, structured recommendation forms provide comparable 
data, from multiple sources or for multiple candidates, but may 
hot be flexible enough to access the unique data the reference 
person may have about the candic3ate. 

The exact reason for using recommendations varies by job, 
company and particular user. In general, recommendations are 
intended to obtain information on job-related skills, employee: 
character, work habits and employment history, it is the most^ 
conventional way to check on what ah applicant says he or she 
has accomplished. A candidate will tell you what jobs he or 
she held, and what responsibilities he or she had. - An employer 
will tell you how well the candidate fulfilled those job re- - 
quirements. _ 



Unfortunately/ since most recommehaatiohs are relatively 
uhstf uctxaf ed, they are consequently somewhat unreliable. Dif- 
ferent supervisors and acquaintances have different levelq of 
writing skill/ thsir .vocabularies differ/ and their skill at 
person perception and understanding of the hew job are highly 
variable. As a consequence, the data they contribute are also 
variable, resulting in interrater unreliability. In addition, 
employers react favdrably to well-written f ecommehdations, re-* 
gardless of the quality of the candidate described. While 
writing skill may reflect something 4bout the basic intelli- 
gence of the author-observer, an employer will have difficulty 
operating the erudition of the reference from the important 
qualities possessed by the job applicant. As a consequence, 
judgments based on reference quality rather than content may be 
invalid, reflecting more on the writer than the subject of -the 

letter of reference (Mosel and Gohen, 1959) . There is also 

... . ... .... 

some evidence that employers stereotype authors of recom- 
mendations by their sex, and judge recommendations accordingly 
(Kryger & Shikiar, 1979). Nevertheless, recommendations may 
include valid, useful information on the applicant's character: 
the. validity of such information is best determined for the 
population and job of interest (Mosel, 1956). 

In sum, when recoinihehdatiohs are used as part of an employ- 
ment process, three issues need to be consider^d^_(JLL-^^ 
empl6y^r_should---r^ogTT^^ unstructured formats are unreli 

able. The user should determine whether to structure content 
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or prescribe reference sources in order to reduce that unrelia- 
bility. (2) Those who actually use the recommendation in a 
decision process should be forewarned or trained to attend to 
content rather than style of the reference, and to consider the 
related validity issue. (3) With the advent of the 1974 
Family Educational Rights of Privacy Act, applicants who submit 
letters of reference from teachers and professors may request 
that' those letters be placed in a file open to candidate' in- 
spection or a confidential file to which the candidate has no 
access. While it i:i unclear exactly what the difference would 
be between two letters by the same author for the two different 
files, it appears that employers react more favorably to con- 
fidential files, regardless of the enclosed recommendations on 
candidate competence (Shaffer, et al., 1976). 

Work Samples/Simulations 

'One method of determining a candidate* s suitability for a -y 
job is to have the candidate i: :y his or her hand at a simula- 
tion, or sample of job tasks. While we may refer to- all such 
tests as work samples, it is important to remember that there 
are potentially as many different .forms of work samples as 
there are different jobs to be fJJJ^d-.— roughly 
-cias^sif iedTinto two groups: motor, involving the manipulation 
of things, and verbal, usually language-oriented or people 



oriented (Asher & Sciarrino^ 1974).* These work sample , 
assessments are usually conducted outside the personnel office 
and away from the actual job situation in a place where there 
will be no interference from factors unrelated to the test 
and where standardized observation of performance is possible. 

Work sample assessments are generally intended to ascer- 
tain the level of specific job related skills that the can- 
didate possesses. Self-report of skills, from interviews,, 
resumes or application blanks, can be verified in work 
samples. Coordination, planning and other cognitive skills, 
as well ais interpersonal and motor skills - hich may be very 
important for good job performance can be evaluated by 
observation of work samples, although they cannot be assessed 
from most paper and pencil tests. While an applicant may 
demonstrate that he or she possesses some knowledge by taking 
a paper and pencil test, a simulation is more appropriate for 
demonstrating an ability to apply that knowledge. 

The reliabilities for work sample tests generally are not 
reported. It is not unreasonable, however, to conclude that 



* The verbal category would include most cognitive tasks, for 
even if the cognitive process involved neither language npr 
people, the output would have to be verbal to be evaluated. 



77 

-1.68- 



they must be fairly high on the average on the basis of the 
similarity between work samples and skill tests* Validities 
are fairly high for work sample tests, an impossibility 
without good reliability. Also, wotk samples are by d>5fihi- 
tion highly structured situations where specific tasks are 
observed and prescribed actions are credited as appropriate* 
However, to our knowledge, no useful reliability data exists 
to support these inferences. Nevertheless, as already sug- 
gested, the validity evidence for the work sample is strong 
in a review of over 60 validity studies for both motor and 
verbal work sample tests (Asher & Sciarrino, 1974) . Two- 
thirds or more of all the- validity studies reviewed exceeded 
a validity coefficient of .30, regardless of whether the work 
samples were verbal or motor, or whether the^criterion was 
job proficiency or training success. More specifically, the 
literature suggests that verbal work samples outperform motor 
work samples in predicting training success, and motor work 
samples likewise outperform verbal work samples for prediction 
of job proficiency. However, Asher & Sciarrino (1974) found 
that work samples of both kinds consistently finish behind 
biographical data in validity, referenced to the criterion of 

job proficiency. . 

These findings raise some issues, as yet unresolved, 
about the nature and use of work samples. . First, ii, as has 
traditionally been argued, a work sample tends to be a valid 
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predictor because of its point-to-point relationship. to the 
job- performance criterion, then why do biographical data 
items tend to be more valid?. Perhaps the focus on behavioral 
matching overlooks the f acilitiating motivational set which 
can be conveyed in biographical data. Second, Weitz & Adler 
(1973) suggest that simulations should be^ short or of 
moderate duration". In long simulations, subjects may start 
to adopt simulation-specific skills whi^iT^re not job-related, 
and in fact interfere with transference of ^he basic work 
skills to the real work situation. To the best of our knowl- 
edge, ho one currently using work samples for selection con- 
siders this caveat from the history of simulation training* 
Third, while different racial groups may perform equally well 
oh the work sample, they may have differential attrition rates 
on the job (Farr, et al., 1973). If turnover is a criterion 
of interest, then a user should do careful subgroup analyses 
of work sample performance and subsequent nontask behavior in 
the work setting. o 

The last, and most basic-dssue is, how does one construct 
txnd score a work sample? Clearly some kind of job analysis is 
necessary to build a simulation with content and face validity 
(Campionr 1972) , but a job analysis that simply lists tasks or 
outcomes is insufficient without a -component that ranks oz" 
rates their importance for job functioning. It has been shown 
th^t a content-valid simulation, the "in^basket" exercise, may 



hot be predictively .valid unless it is scored oh the basis of 
those skills which are most important for the job (Brass & - 
Oldham, 1976) • 



Thought Samples 

A special technique somewhat similar to work sampling has 
been developed to assess, in particular, motive dispositions 
related to various jobs. It involves objectively coding 
samples of a person's thoughts in imaginative stories written 
in response to pictures. The codes ^ were originally derived 
(see McClelland, et al. , 1953; Atkinson, 1961; Winter, 1973) 
by identifying what characteristics of thought regularly 
appeared when a given motive was aroused. Then, if those 
\ thoughts occurred in stories written by subjects hot under 
conditions of motive 'arousal, it was assumed that they were 
generally under the influence of the motive which uniquely 
produced such thoughts. In this way, measures have been 
developed for the need for Achievement (McClelland, et al., 

1953), the need for Affiliation (Atkinson, 1958) and the need 

- _\ - & 

for. Power (Winter ^ 1973). 

The general logic of this approach assumes that being 

concerned about certain issues, or thinking a lot in terms of 

certain goals, means that a person will act in ways 'that are 

especially appropriate for-success in particular jobs. The 

logic is not unlike that for woiJc samples: what^a person does 
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(thinKs) in a sample situation predicts what he will spend 
his tirae^^ing' on the job. Two ^notivational thought patterns 
in particular have been associated with successful vocational 
' .outcomes. The first involves. the need for Achievement, or 
the tendency' to- think a lot about doing things well or in a 
^more efficient way. Such -a thought pattern has been regularly 

.'found to be assocfiated with- success as a small businessperson / 
^ .. _ ■ --------/. 

or entrepreneur (McClelland, 1961, 1966; McClelland & Winter 

1971; Muiti & McClelland, 1979). The relationship has good /' 

theoretical validity because it makes sense that a person/wno 

thinks a lot' about doing better, e.g., getting more output 

fcJr less input, should be just the kind of person who will 

succeed in a small business which requires constant attention 

to inpat/output ratios. 

tfie Second motivational or thought pattern is associated 

with nianagerial success in larger businesses. It is called 

the leadershi"^ motive pattern and involves "a relatively high 

- need for Power which is higher than the need for Affiliation, 

and a high sense <?f ijelf-control . This motive pattern has 

been found to be related to success as a sales manager 

(McClelland, 1975), to rated performance as a Nayal Commanding 

'and Executive' Officer (Winter., 1979) to promotion to higher 

levels of management within "^he American Telephone and 

Telegraph Company over a 16-year period (McClelland, et al., 

Ij and generally to success in top managemerit jobs in 
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American companies (Boj^atzis/ 1979) . Again the relaticn§i^ip 

J ^ . • ^ - 1 

has good face vaxidi^y, for,i*t means people -^who make good . 

- - . - — £ • 

managers tend to think a lot 'about . influencing others, care- 
fully control their influence 'attempts, and are not exces- 
sively concerned whether they, are liked or disliked (need for 
^ Affiliation) . 

The reliabilities for , measured obtained from objective 
coding of thought samples .have generally been reported to be 
low (Entwisle, 1972). However, there are important reasons 
for believing that the coefficients reported -may be serious 
underestimates of the true stability- of the measures. As 
McClelland has pointed out (1971) , ^thfe, instructions for the 
test tell the subject to "be^ creative, " which is interpreted 
by them to mean that they should -tell different stories each 
time. It is well known that in all or*ganisms there is a 
btfT^-ih tendency to vary spohtaneptis responses, which has 
been called "associative refractory phase. " Winter and 
Stewart (1977) have demonstrated that if the variability set 
is broken by telling subjects on the second administration of 
th»B picture-story test that they are free to tell .the same or 
different stories, then test-retest coefficients, rise to the 

more respectable level of .60. More importantly, the validity 

- ' - - *« • - 

studies Mentioned above indicate that the measures must have 

a higher reliability than what has typically been reported, 

if* we assume that validity cannot .be higher than reliability. 



Finallv ,'^tkihsoh and Birch (1979) have argued that for 
measures of spontaneous behavior like these, the traditional 
psychometric model of reliability does not apply. — 

These measures have an importr^nt but limited utilityl 
They are objective in the sense that the coding schemes for 
them are precise enough for two different judges to obtain 
a high degree of agreement (r=.85-.90) in coding the same 
protocol. ?or types of positions in ehtrepreneurship and 
management for which the most research using them has been 
carried out, they have good face validity, and they do provide 
information on the motives needed for these two types of jobs 
which is not obtainable in any other way. On the other hand, 
their limitations are: (1) that the testing conditions must 
be carefully controlled since stories written can easily be 
influenced by situational factors? (2) that they are more 
costly to score than tests involving machine scoreable 
choices; (3) that their coverage of competencies and types of 
jobs is so far quite limited; and (4) that the method is often 
viewed with some suspicion because it appears to be getting 
at some unconscious aspect of the self. 

Nevertheless, the principles behind thought samplihg-- 
objectively coding thought patterns — can be extended to 
selection techniques such as the interview to determine the 
response of a wider range of skills. McClelland (1973) and 
Boyatzis (1979) have reported success in isolating and 
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reliably coding evidence for intellectuaal- arnd' interpersonal 
competencies' iri7 top performers through unstructured self- 
report. These competencies were then made the focus of 
specific psychometric tests which successfully validated their 
presence in better performing job incumbents* 

Detectors of^>ec^-pt.ion 

' One group of employment prescreens seldom studied by 
psychologists are the detectors of deception. These include 
the polygraph, voice stress analyzer, and paper and pencil 
methods of detecting deception. Each of these methods is used 
occas|pnally as an employment prescreen by organizations con- 
cerned with employee theft or confidentiality of corporate 
information^ The polygraph is designed to measure physiologi- 
cal indicators of stress such as pulse rate, relative blood 
pressure, rate and depth of respiration and galvanic skin 
response (GSR) . Voice stress analyzers presumably measure 
inaudible stress-related frequency modulations in the voice. 
Stre:^s is assumed to increase when the applicant is trying to 
deceive the tester. The paper and pencil instruments are 
designed to access information about applicant attitudes 
toward thefts admissions of theft, and biographical correlates 
of deception. The actual questions included, equipment in- 
volved, and time spent oh detecting deception may vary between 
different organizations but all of these employment screening 



processes are administered in the employment office or testing 
center. Administration would appear to be under constant 
conditions within an organization. 

Strictly speaking, detectors of deception do not try to 
identify any job related knowledge, abilities or skills. If 
an applicant should try to deceive the employer about his or 
her background or experience, this would presumably be de- 
tected as physiological or vocal stress. The employer, 
however, appears to focus the examination on issues related 
to honesty, theft, and lying behavior rather than on issues 
related to job-related skills. 

Psychometric evidence of reliability and validity is 
tenuous foi these employment prescreen methods. Reliabilities 
in the .80s and .90s have been reported for the polygraph 
between graduates of the same training program, but reliabil- 
ity drops signif icahtiy for more heterogeneous groups of 
polygraph operators. Likewise, the percentage of accurate 
response (validity) reported for polygraph use is presentable 
but the research designs do. hot correspond to the realities 
of employment situations (i.e., -^accuracy is assessed in^ 
circumstghces when all subjects have some; piece of information 
to hide and accuracy^^is equal to the hit rate for an operator 
in identifying the deception). In most employment situations, 
the operator doesn't - even know what areas of the individual's 
history include deceptions, and very few applicants in the 



total applicant population may have some empl6yineht-r elated ; 

problem to hide. Demonstrated validity fojo_voij^_jstress^ — 

analyzers is even lower, and again, is determined in situ- 
ations hot analogous to employment settings. Sackett & Decker 
{1979) review the reliability arid validity data on all three 
techniques in some detail, and conclude that the techniques 
are fairly widely used in criminal investigations and employ- 
ment selection decisions, but th%? validity data that exist 
are inadequate to support the use of these, methods for 
selection. 

In sum, most of the comments that can be made about the 
use of detectors of deception are 'in the form of cautions* 
Before implementing polygraph tests. as employment prescreehs, 
one should be sure to ch6ck state law* CJse of the polygraph 
is currently restricted in 15 states, with 19- states requiring 
licensing for all polygraph operators* ' There are also ethical 
considerations in the use of such devices. Is the polygraph 

-an invasiQn of privacy, as is argued about personality tests? 

* _____ " _ __ _ _ _ _ _ . 

Can one decide that someone is lying on th<5 basis of a test ^ 

of unknown validity in the employment setting? These two 

questions bring us to' an is£^e at the heart of any application 

of detectors' of deception: Are these . me.thods valid in an 

_ «• _ _ . . . _ _ ■ ■ ._ _ 

employment setting where base rates of deception are lo\j? 

Virtually all research on the reliability and validity of 
detectors of deception has been conducted in actual or simu- 
lated criminal investigations ; ^ The abili^^ of these data to 



be generalized to low base rate employiheht situations is, 
as yet, unproven. 

Table -9 summarizes reliability and validity data on 
application blanks, resumes, recommendations, _ work samples 
and simulations, thought samples, and detectors of deception • 
These data are less voluminous than those for the interview 
and the psychometric test, and ar'e presented together for 
ease of comparison. 

Differential Validity and Test Bias 

^ When considering .the merits of using any one of the pfe- 
ceding selection methods, whether from the compliance perspec- 
tive of a government agency or the pragmatic perspective of an 
employer, a primary concern is whether *:heir use. will result 
in adverse impact on protected classes of applicants (i.e., . , 
subgroups identifiable by race, creed, color, sex, religion, 
national origin, age,: marital Status or handicap). We define 
adverse impact as disproportionate hiring of individuals on 
the basis 'of their potential job performance. For example, 
when a hospital accepts applications and; selects chaplains 
who are mp.le and Catholic only, this does not constitute 
adverse impact orh members of other groups, because being male 
and being Catholic are bona fide occupational qualifications' 
for a position which requires a person to give the las-t rites 
to Catholic patient. . ^)ther individuals • are , excluded on the 
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Reliability And Validxty Data By Technique 



Method 


Reliability 


Validity 




Study 


EsLtima,te 


Method 


^saff iciest Method 


Criterion 


APPLICATION 


BLANKS 








Goldstein 
(1971) 


43% CO 
85%' de- 
pending 
on item 


agreement 
between 
applicant 
data and 
that data 










verified 
by checking 
with past 
employers 




0 


Cascio 
(1975) 


.4l£r£ 
l.d, me- 
disui r= 
.94 


correlation 
of applicant 
data and 
'Verified data 






^Kisating, 
E. et al ' 
(1950) 


r=.90 to 
.98, mdn= 
.94 


correlation 
of employee 
and employer 
data ^ 


• 




Mosei & 

Cozan 

(1952) 


r=.87-.98, 

mdn=«#94; 

mean 

agreement 
on job 
duties 
85.5% 


correlation: 
of -applicant 
and employer 
data oh work 
history; job 
duties com- 
pared by % 
agreement 






Roach 
(1971) 






.29 predictive 
criterion 
related 
validity 


tentire in 
montlis 


Asher 
(1972) 
' (review) 






55% of criterion 

reviewed related and 

validity cross vali*- 

coef fi- dated 

cients are 

greater 

than. or 

eqttal to 

r=.50 


job pro— 
^.ciency 



*work history gathered by interview method 



(Table 9, continued) 



Method 
Study 



Reliability 
Estimate Method 



APPLICATION BLANKS ( cont ' d ) 



Helmreich 
et al 
(1973) 




• 


.59 


cross vali- 
dated; cri- 
terion re- 
lated vali- 
dation (pre- 
dictive) 


et al 
(1974) 








related 
(concur- 
. rent) 7 not 

cross vali- 
. dated 


Nevo 
(1976) 






.36 

(menl ; 
.18 

• (wome:.) 


criterion 
related 
cross 
validated 










(concurrent) 


Hinrichs 
. et al 
(1976) 


.78^ 

.75:"- 

.65 

.77 

.58 

.75 

.49 


KR20 

internal 
consistency 
within eact 
sample 


.72^ 
.72k 
.42 
I .56 
.24ns 
.38ns 

.26 


(concurrent) 

criterion 

validity; 

cross* 

validated 



Validity 
Coefficient Method 



success in 
Navy diver 
training 



art vs . non-- 
art student 



military rank 
at discharge 



pooled overall 
ratings by * 
three execu- 
tives 



*ail correlations are significant (£ < .05 or better) unless otherwise 
indicated (ns) k-samples used to generate scoring key 



RESHMES 



V.O DATA AVAILABLE 



REGQffilENDATIONS 



Mosel & 
> Goheen 



(1958) 



4_of 12 
validity 
coeffi- 
cients 
for 12 
jobs ^^were 
significant; 
.2l^r;^.29 

89 



(predictive) job proficiency 
cr i t er io h f rom sup er • 
related vised perfor- 

mance ratings 



EKLC 



-1.80- 



(Table 9, continued) 



Method _ Reliability. 

Study EiStimate Method 

WORK -S3aS>tES/SlMg£ftTI0SS 

Asher & 
Sciarrino 
(1974) 
(review of 
60 studies) 



Validity 

Coeff icient Method Criterion 



% of r< 
exceeding 
the given 
value 



r 


motor 


verbal 




.30 


78% 


60% 




.40 


70% 


41% 


job pro- 


.50 


43% 


21% 


ficiency 


r 


motor 


verbal 




.30 


79% 


81% 




.40 


47% 


65% 


•training 


.50 


43% 


39% 


success 



DETECTORS OF DECEPTieS 



Sackett & 
Decker 
(1979) 
(review) 

* Po ly graph r = . 8 G 



interra- 
ter reli- 
ability - 



80-90% 



accuracy 



Voice Stress 
Analyser 



Paper & 

Pencil 

forms 



median 
accuracy 
reported 
= .30 

median ' 
r in • 
low 40 's 



compared to base rate 



known base 
rate of 
guilt or 
to exp^t 
opinion 

compared to 
known truth 



correlated 
with theft 
admissions 



or expert 
opinion 



kiiown truth 



theft ad- 
missions 



*Note: These figures make assumptions of base rate of dishonesty which 
are .unre^.listia and inappropriate for polygraph use in employment 



basis of their sex and reiigion but that exclusion is job- 
related. If, however, a hospital refused to consider or hire 
female applicants for the position of P^^ 

their actions would have adverse impact on women since both 
m^n and women can be ordained ministers in most Protestant . 
sects. The exclusion on the basis of sex would hot be job- 
related. 

Adverse impact is the visible consequence of what psycho- 
metricians and industrial psychologists call test bias. In 
this phrase the word is used in the very generic sense adopted 
by the EEOC and other federal agencies charged with assuring 
use of fair employment practices; a test is any paper and 
pencil or other measure used as the basis of an employment 
decision. In th# preceding examples the implicit, question, 
"what sex are you?" is the test used as a basis of the 
employment decision. The phrase test bias is used to refer 
to unfair consequences of using tests (predictors) as the 
basis of selection decisions in a real life situation. A 
test which is used to disproportionately eliminate applicants 
on a non-job related basis shows test bias. The most visible 
and socially unacceptable examples of test bias disqualify 
•minorities, the handicapped, etc., from jobs on the basis of 
predictors (tests) that. are. not in fact job-related. Test 
bias may also occur but result in no adverse impact. For 
example, as. an employer, one might arbitrarily decide to hire 



only receptionists who indicate oh their applications that 
they aire left-handed. This would cbnsititute test bias since 
being lefi handed is tiot job related. It would not, however, 
constitute adverse impact since no protected classes would.be 
disproportionately eliminated frara consideration; all right 
handed people regardless of subgroup membership would be 
disqualified. This kind of test bias is not illegal under' 
Title VII of the Civil Rights Act of 1964. 

For the afposes of federal agencies, employers and the 
law, test bias resulting in adverse impact is a serious prob- 
lem and the driving issue behind compliance reviews, careful 
choice af selection procedures, and court cases. Therefore, 
given the preceding definitions and the concerns of our 
readers, we will focus now on the potential for. illegal test 
bias (i.e., adverse iinvact) from us^ of selection predictor 
measures. 

Since test bias refers to unfair consequences of test 
use it is not surprising that several different authors have 
attempted to define what is an unfair consequence. As a 
result there are now nearly as many different models of test 
bias, defining unfair consequences in different ways, as 
these are authors who write abo..t test bias. For example, 
there are 11 models of test bias included in Peterson & Novick 
(1976), each of which defines fair use of a test differently. 
Basically what these models do is define what kinds of 



selection errors must be minimized for a test to be fair to 
all applicants. Some authors say ^hat a test must be used tc 
maximize choice of individuals who will succeed; this is a 
fair use of a test. Others say a .'test should be used to 
minimize the likelihood of either rejecting someone who would 
have succeeded, or accepting someone who would have failed. 
Still others try to maximize the chances of rejecting a person 
who would have failed. Even though these do not? sound like 
contradictory goals r each definition of what constitutes fair* 
use of a test implies dif f erent statis-tical corr^tions ^should 
subgroup performance differ on the predictor. Measures taifen 
to prevent test bias vary greatly depending on the mod^l ac- ^ 
cepted by an organization. To use a test fairly an orgahi^a- 
tion may choose appropriate cut-off scores and select propor- 

_ y _ 

tions of individuals from subgroups so as to maximize its goal 

in termi of fair consequences while miriimizin^honjob-felated 

\ . _ .. __ x' - 

disadvantages to members of subgroups. 

The TTniform Guidelines on selection practices, as well 

as thei . edecessors , have focused on adverse impact and in 

particular on differential validity leading to adverse impact. 

The term differential validity refers to a s.ituation where 

the correlation coefficient between a selection procedure 

(e.g.f test score or information used as a predictor) and job 

- - ; X 

performance is significantly different for different appli-^ 

cant subgroups. In cases where a difference in these 

93 



validity coefficients exists between subgroups identifiable 
by racer ^ex\ etc,, but the test score or predictor inforraa- 
tion is used in. the same way to select applicants without 
re^iard'^to subgroup differences, test bias and adverse impact 
results. An organization that does not closely Examine its 
selection procedures for differential validity may end up 
using its selection procedures in ways which have' unfair 
consequences. However, once a R:Cdei of test bias has been 
chosen which targets unfair consequences of test use, statis 

tical cor rect i oris c^h be made and selection cut-off . scores 

l_ 

set to reduce or eliminate adverse impact /from use of the 

, I \ 

predictor test. Therefore, tl^ough differential validity can 
obviously lead to adverse iinp^t, it rueed^nQi: do so. The 
potential unfair consequences of differential validity can b 
statistically eliminated while still allowing for use of the 
test. 

The explicit attention given to differential validity 
as a cause of adverse impact has blinded many usets to the 
broader issue of unfair test use. (i.e., test bias resulting 
in adverse impact). It is very important to note that an; 
absence of differential validity does not preclude adverse 
impact. GoriseqU'.mtly, ah organization which assuities it is ^ 
using a test fairly because correlations between tests (or 
predictors) and performance are equal across subgroups may 
be in error. If the range or means of predictor scores for 



> ^- ... • • 

those subgroups differ, setting predictor score cutoffs to 
select applicants as if the groups were identical will result 
in test bias; selection without attention to characteristics 
of the .subsample scores {including mean, range or validity 
coefficient) may also result in adverse impact. • 

Adverse impact, therefor'^, results from test bias and may 
the consequence of any number of factors, of which differ- 
ential validity is but one. Thus, as suggested by Fincher^ 
(1975) and Schmidt, Berner and Hunter (19731, differential 
validity may be a pseudo problem that overshadows the problem 
of test fairness. 

Early court cases concerned with discrimination resulting 
from use of employment tests focused on test bias and differ- 
ential validity without clearly distinguishing between the 
two. In the classic Griggs case the courts disallowed the 
Duke Power Company's test because they were not demonstrably 
job-related for the blacks' who we^e required to take them to 
^ advance, and because they were used in a way that had clearly 
negative consequences for the black workers, The impression 
gathered from our sample ^-f employers, is that organizations 
have interpreted the ensuing court cases and public policy as 
being solely concerned with identifying cases of differential 
validity -ieven though, as indicated above, differential 
validity is hot necessary for test bias cr adverse impact. 
Nonetheless, policy-makers in business and government have 
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been rightly concerned with the need to conduct subgroup 
analyses and- be wary of the adverse impact that may result 
if generalized -decision rule for^hiring is used to select 
applicants from sub-groups whose validity coefficients difffer* 
Gorporate experience and acad^ic research have demonstrated 
that differential validity sometimes occurs •* More importaat 
issues are: . when and why differential validity occurs; how 
the controversy^ over the existence of differential validity 
relates to the central issue of test bias; and what steps are 
possible to prevent adverse impact of test use- 

We refer to recent, writings and opinion on differential 
validity as a controversy because the current trend among 
researchers is to aggregate individual studies of differential 
vais-dity and draw conclusions about whether differential 
validity in fact exists at ail I Researchers and practitioniers 
who ha"^e calculated subgroup validities and found signifi- 
cantly different validity coefficients may be very skeptical * 
of any research which questions the existence of differential 
validity. Nonetheless,, a few researchers have examined large 
numbers of studies in an attempt to detexniihe if firdihgs cf 



* For a compete review and bibliography of all such pubiished 
' studies of differential. validity, see reviews by: Boehm, 
1977; Kat:3ell & Dyer , 1977; Dunnette S Borman, 1979; Hunter 
Schmidt & Hunter r 1979; Arvey, 1979.) 



differential validity are chance occurrences or are indicative 
of true differences in subgroup test performances. In one 
such investigation Boehm (1977) examined 31 studies involving 
297 comparisons of employment and training selection proced- 
ures administered to blacks and whites. Of those 297 compari- 
sons only 8% showed differential validity. A closer Iook at 
those studies revealed that reports of differential val^riity 
were likely to come from methodological weak studies, L^^., 
those characterized by small sample sizes, criteria whic-: wer~ 
not performance measures and/or a weak rationale for hypc-r:<> 
sizing that the predictor (s) would hiave any relationship r- 
the criterion. In sum, she concluded that findings of differ 
ential validity were methodological artifacts^, and did not 
reflect any relationships across subgroups. In another study 
Katzell and Dyer (1977) reviewed 31 studies and cc^cluaed 
that differential validity did occur at above chatic:?^ level^ 
in these studies but that differential validity die not fav" 
one suogroup over another (i.e., validity coefficients were 
lower for whites as often as for blacks) . Finally, in an 
extensive study aggregating 866 validity "comparisons of blac<3 
and whites from 39 studies. Hunter, Schmidt and Hunt:er (1979} 
identif ed and controlled for methodological biases. They 
conclu -d that findings of differential validity were procurred 
by eha r and statistical artifacts, and that therefore dif-^ 
feren* validity probably did not exist in the population. 
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If testing practitioners f i^rrd differential validity when 
examining their selection procje^cares but researchers ar-rue that 
-differential vallditi ;?ocsn't : ally exist, what can we conclude 
:z30ut differential validity a I tiie potential for adv&rs impact 
: - employment se^^^utn^on? The ^ecent research sucgests rnat 
—areftxl design validation s*^.:dies may prevent Siffer'^tial 

Talidi^y from oc^ r :.r.c . ice -singly careful vajJ-daticrr, con- 

rrol, .^nd desicr. ziay elinina' differential validity asr a source 
adxeiT^ impecii ^weve::: , ^ ^ust also remember tha": e3 imina- 
cL different::.^ v2l.\f:rrj.' .s not a panacea because many 
r-rr^r rhacacteris tics cd rr^ i^c^r ibution of sub^mrrle predictor 
~3cre:E r ~y coh^-rdSate r5 hrrns and adverse :^arnac^^ . fione^the- 

isz if -^Af. y designed idation studies ar e coTsSartec to 
r^fLiire ti:B pos^rrility of '-^ncirnr differential va^/idirVj, then 
rnese st^ies vill als:; provide i^ahs, standards :=^viHnicns and 
icore ranges wf^ can be exa^i^^i to determine :=^eii protentxal 
for adverse xjnpaet. 

The difference models whicr identify unfair u» oi = test, 
and provide a ^anionale and method for setting cuttinc srores 
i.nd selecting Lr^ividnals fairly from differing subgrrcLzrs, can 
.:e used in ccrrthL-Tiaticn with a good validity study to guide an 
•organization ^rsward more job-related and equitable selection. 
3^!:odels of test cias reflect selection goal^' of suf f ici.=errt 
diversity tc ^f^r-bmmodate almost any organizational chc^.Te about 
^he kinds of ^ r^ ^inn errors that constitute test bias„ One 
^odel (Darli r r ^.— ii y cited in Peterson & Novick, 1976) ^rv^er: 
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exi:st:s which sl^ws orgHnizatiohs tfiat wish to consider r^J± cnly 
pec^arssiice cnrssria, but also employment parity or ccanpsnsatrory 
hi:r±ig of mincrirries in their definitions of test fairn^^- 
This model pennies explririt inclusion of such factors in string 
curof f s 2nd making selec cion decisions which are fair in accord- 
apc3 with the ar=anizati -:;n ' s values. Regardless of intentu^ons 
ancT- c.^rcir-E of T ^y regi^ding test fairness, an orgamizaxion ' s 
r^esix-ces eh:- £e inadequate for a good validation study because 
saii^pZe sizes i.,e., jcb cp«rings) are lintited. For this reason, 
•;:ooperative validity generi^ization studies make the vaiidatiori 
r_ocess an ever mere pala'^^^i^le strategy bemuse they can provide 
a^ta to reduce adverse ^ and enable crr^-er work force 

oroducti vit- at ^^:ii±ucec u:^ :s. 
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Comparison of Selection Devices 

After a Vc^riety of techniques have been describes and their 
research reviewed, it is important to step back and put them 
into perspective, Fecoimendations can be made, endorsing 
certain methods' for certain purposes • Table 10 is a tool for 
doing just that. The techniques are compared on the basis of 
eight important issues for someone choosiTig a method for per- 
sonnel selection: (1) reliability, (2) validity, (3) research 
support, (4) objectivity, (5) face validity, (6) unique data, 
(7) cost, and (8) skill coverage, Descite the wide variance in 
the amount and depth of research oh each technique, objectiviry, 
cost, skills covered, and psychometric properties; each of the 
data collection methods is appropriate to use for certain 
purposes, ^ 

The application blank is commonly usee by most companies as 
it is- an inexpensive means of gathering background data, in- 
cluding experience and education. Applicants expecJ: to fill 
out an application blank, thus face validity is good. Addi- 
tionally, reliability and validity of certain items can be 
juite high. The disadvantage of the application blank^ is that 
it does not directly measure skills. Therefore, where assurance 
of skill is critical, the application blank would be deficient 
by itself as a data ba^e for an employment decision. When the 
employer has little doubt about applicant abilities (e.g., 
applicant has a license or degree indicating skill level) 
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Comparison of ai Techniques 



. _ S(M.fr FACE ONIQDE 
METHOD pgiABttlTY mhm RESEfiRCE SlBfcgJ?.--- ^Umm 



COST 



Intsr- 
views 



tieneial 
itatelli- 
gence 
Tests 



Specific 
Skills 
' and 
ftbiiitys 
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very specific very sge- extensive, 
to .situation cific to primarily 
and inter- situation focused on 
viewer; Ma^sn s inter- interper- 
Range=.55-.rn viewer; Me- sonal per- 

dian Range ception snc 

=.20-; 25 decision 

for JOB fonnulatior: 

PROPICIENCI; 

somewhat 

higher for 

training 



Good; Higher va- extensive 

Median lidity for 

Range=.80-.90 managerial 
and clerical 
occupations. 
Median Range 
.=.20- JO for 
JOB PROFI- 
CIENCY; some- 
what higher 
for training 
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Spedfic Higher va- extensive 

to instru- lidity for for most 

ment; Median managerial ability 

Range=-65-.80 emd cleri- tests. Very 

cal occupa- little on 

tions; He- measures of 

dian Range creativity 

=,25-. 35 and judg- 

for JOB meiit 
PROFICIENCY 
somewhat 
higher for 

frai 



scjtfCtive 
er 



scarin? 
keys 



rel 
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key- 



osually 
good, de- 
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interview 
antent 



provides moderate insHecaL 
opportunity to hiix, abL-ity, aov 
to interact depeadiag aotaati x iter- 
with appii- on aacEont pessona" :y*^ions 
cant and inter^^w- 
observe in- er tie, 
terpersonal aamber 3f 
dynamics interest 
and trsi-ring 



fair depen- provides 
ding on job overall 
ability 
measure- 
ment 



usually Gerwr i ^•"jong 

low witr. & LiSjiC; ^e: .bi 
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dardizEC fHasr. v; imf^ical 
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crease ja floe^cv; SaiiM 

costs vcdi Relat^ais 

adminis^ 
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usually provides low adarar S^icarate measures 
good depen- independent istracaon- of- Verbal ability, 
ding on job measure of cost; ind- ftwt^^al iJ^ty, 
skills and crate *r Sjft-tdal' flbili^, 
aptitudes, high Cic^vity, S^cho- 
equCTP aotar Ability 
cost£ |jj2 
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on items, 
discrimi- 
nation, 
bias 
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structured 
questions are 
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Subjective 
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procedures 
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self- analysis, 

perceptions keying, 
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weaknesses validating 

and 

strengths) 
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self- 
perception 
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ex-boss and cost 
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call or 
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letter) 



level, communication 
ability 



job proficiency , 
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and where cost is an issue, the applica-tioo' blank can be a 
fairly inexpensive and effective selection tool,*"" 

If ah applicant's goals and self-image are important, the 
resume is a ^less structured way to access and expand on this 
information. The cost to the organization is minimal and a 
resume provides more 'detail in these areas than does an appli- 
cation blank. Little research evidence is available on the 
reliability and validity of the resume. Like the application 
blankf the resume will seldom be sufficiehc for making selec- 
tion decisions as skills are not directly measured. 

In - those .cases where interpersonal skills or task related 
skills are important on the job; recommendations may be a 
useful addition to the selection process. These are inexpen- 
sive and provide a subjective measure of skills which are meas 
ured more objectively/ but also at greater cost, by other 
methods. There has, however, been little research oh the 
psychometric properties of recommendations. 

The work sample appears to be an .especially useful aata 
collection method as it can directly 'assess intellectual, 
yiriter personal, err psychomotor skills. Psychqjnefer ie data show 
work samples to have high validity anc3 good consistency rela- 
tive to other techniques. The cost may be prohibitive. 



* Though employers often worry credentials such as li'behses 
and degrees say something about an applicant's competence. 
Chapter 3 examines this questionable assumption* in great 
detail. 



however, as work samples generally require large capital expen- 
ditures and may be expensive to administer. 

Thought samples provide some of the most useful data avail- 
able in assessing these institutional components of interper- 
sonal competence. This technique , indeed, is the only way in 
which motives can be measured. In complex jobs in areas such 
[ as management that rely oh interpersonal skills, thought sam- 
ples have shown the greatest consistency in predictive validity 
over time. They are. expensive to'score properly, but return a 
long-term selection benefit far out of proportion to the ini- 
tial scoring investment. 

Detectors of deception such as polygraphs have little to 
recommend them. Their reliability and validity are tenuous; 
they tend to antagonize the subject. Nonetheless, in situa- 
tions where the good will of applicants is not a consideration, 
where false negative decisions (i.e., identifying an honest 
response as a lie) are unimportant because of the tremendous 
-^cost of false positives (i.e., hiring a dishonest worker), then 
polygraphs (and such) may provide some unique, data of utility 
for that specific decision situation. 

Interviews have high face validity in employment situations 
and may be particularly useful iiT'asse^sing interpersonal 
skills. Though reliability is typically lower than many other 
measures, with only moderate expenditure it can be raised quite 
"substantially. Validity is low when the interview is used^o 
assess skills other than interpersonal. Therefore, when other 
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skills are required, the interview will be ah inadequate meas- 
ure. The objective coding of interview data as discussed under 
Thought Samples provides the best potential solution to prob- 
lems of reliability and validity. Because of the time required 
to administer an interview propelWLyr its cost may be prohib- 
itive for its utility as a selection device for many jobs. 
However , -if - irt ~i^-^^^ as well, such as 

public relations, the cost may be more easily justified. 

'ianeral intelligence tests may be appropriate as measures 
of overall intellectual ability for lower level clerical or 
^ managerial jobs. However, specific tests of' intellectual 
ability show approximately, the same cost with higher validity. 
Cost for both measures will be low if a standardized commer- 
cially available test is used.^ 

Specif^ skill and ability tests are available for asses^ 
sing intellectual or psychomotor - skills . These testis usually 
appear iace' valid to the applicant and show good psychometric 
properties. If tests are constructed by the organization for 
particular jobs, costs will increase substantially. Tests may 
also show higher validity for particular jobs if the test uses 
actual job equipment, but, of course, this will also increase 
the cost to the organization* 

Finally, personality tests show relatively low reliability, 
predictive validity and face validity but may be' especially 
useful in assessing interests. AlsOT'^f psychological adjust- 
ment is an issue, these tests may be quite appropriate. The 
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cost of commercially available tests is usually low, but 
administration time varies quite widely, and may serve as a 
source of hidden cost. Scoring keys for particular companies 
can be constructed for many tests, '^his will increase the cost 
but will usually raise the validity of the test as well. 

In sum, all the selection techniques discussed, even the 
most questionable ones, have some advantages. Their . unique 
strengths and weaknesses (in terms of the 8 criteria identified 
above) should all be considered before making a choice of 
selection method (s). 



III. THE PRACTICE OF PERSONNEL SELECTION 

The previous section has described and compared the major 
employee selection techniques currently in use. The present 
section is devoted to the application of these techniques in 
job situations. Nearly all the literature on employee selec- 
tion deals with the selection instrument as the unit of analy- 
sis, rather than with the job or type of job in regard to which 
a selection decision is made. The intent of shifting ettention 
to the job as the unit of anailysis is to examine the appropri- 
ateness of a selection system in which multiple sources of data 
are considered to identify the most appropriate job candidates. 
Given what is known of the strengths and weaknesses of indi- 
vidual selection techniques, ah account of how these' techniques 
are used as the basis for job selection enables the analysis of 
how organizations carry out matching applicants to jobs. 

This section summarizes an empirical study of employee .se- 
lection practices in the field. The present study was intended 
to involve a representative sample of jobs to which selection 
devices are routinely applied, so that the state of the practice 
in entry level . employee assessment could be ascertained for a 
wide variety of competencies. This study was initiated by 
defining a" sampling strategy based on a comprehensive but simple 
job taxonomy which would yield to quantitative analysis. Using 
the taxonomy as a guided direct contact was initiated with more 



than 100 organizations which were expected to represerrr the most 
conrpirehehsive employee selection practices, and to elicit from 
them as much information as they were willing to share regarding 
their current and past procedures. Following the description of 
these procedures, this section presents ah account of the state 
of the practice with regard to key issues in employment selec- 
tion and a statistical analysis of data gathered on 239 jobs 
that documents current trends in employee selection. 

The Scope of the Study 

A prime requisite of this research was a job sampling 
strategy which would be representative of a cross-section of 
job functions, since it was expected that the majority of formal 
competency-based selecton systems would be founcJed on ,a rational 
analysis (e.g.. Fine & Wiley, 1971)* of the tasks and functions 
performed by individual jobs. The functional taxonomy developed 
by^Katz and Kahn (1978) was particularly appropriate for the 
present purpose. These authors described their formal taxonomy 
in terms of five sub-systems which identify formal opeta^tiohs 
within= organizations, and which may also be applied to describe 
dominant job functions: 

1. Production joJ:^ are based on task accomplishment through 
t^chn ical_prjo_f_lc.i eacgL^ These_ar.e jobs i n w hich energy ~ is- -tr-afts— 
formed into output and value is added for the organization. 

2. Maintenance jobs are oriented toward maintaining stabil- 
ity and predictability within an organization. This may take 

ill 
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the form of preserving existing relationships among other indi- 
viduals in the organization, or toward the preservation of the 
^atus quo, 

3. MLahageriai jobs contain the controlling or decision 
making aspect of work. People in these jobs coordinate exter- 
nal requirements with internal resources and resolve conflicts 
among other job functions. 

4. Boundary jobs are characterized by their function as 
linking the parent organization with their counterparts in 
other organizations. Individuals in these jobs carry out the 
transactions leading to the procurement and disposal of goods 
-and services. ~ 

5^. Adaptive jobs are characterized by their roles in or- 
ganization change, including intelligence gathering, research ^ 
and development and planning functions. Though these jobs are 
concerned with innovation , their primary task is helping an 
organization adjust to a changing environment. 

The translation of this functional taxonomy of organiza- 
tions into a taxonomy of jobs, however , involves adding the 
level of skiil with which a function .is required to be per- 
formeS as an independent dimension. Accordingly, jobs were 
also classified according to whether applicants were relatively 
unskilXed with regard to the function they would perform in the 
job; were skil led or moderately proficient in technical and 
procedural aspects of the job, or were professional in their 
skill level, meaning that extensive specialized training or 
experience was acquired prior to selection. Classification of 



each job in the taxonomy by skill level was made on the basis 



of skill level required of the applicant at point of entry into 
ah brgaiiizationr rather than oh the basis of a skill acquired oh 



the job following a suitable period of on-the-job experience or 

0 

training provided by the hiring organization. 

The next task was to identify employers which had documented 
their selection systems sufficiently well to be helpful in the 



present study* it was expected that many employers would resist 
sharing recent documentation of employee selection procedures 
for reasons bearing on the proprietary nature of selection pro- 
cedures r the maintenance of confidentiality of the information 
source r- and qu estions of compliance with EEOG guidelines. To 
minimize this problemr the first approach taken was to identify 
specific individuals within target organizations which had con^^:^ 
ducted selection research and had presented it in both published^ 
and unpublished literature, and ohher individuals -who were 
otherwise known to the authors thrpugh personal or professional 
contact.* ^ ' 



The identification of contacts through the literature review 
produced mixed resultsi^ A significaht number of organiza- 
tions who are pursuing research on their own s;election\ sys- 
tems could be traced through articles published in academic 
journals-; ,^ However r surprisingly, little of value regarding 
the state "of the, practice was gleaned from a survey of peri- 
odicals devoted to personnel practices in specific trades^ 
careers or professional associations. From a random sample ' 
of over 100 -sucS periodicalsr hot a single article within 
the last f.ive y^ars could be found in which either a rigor- 
ous statistical' ahalysis^.of selection sysiem practices was 
undertaken or in which data-based research regarding a par- 
ticular -selection device was attempted. Of those articles 
which dealt with personnel selection pr^.ctices, the only 
data available were anecdotal in nature and notr- useful to 
the present study. 
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This procedure obtained access to selection system data oh 
154 different jobs, from 79 organizations. Ah additiohal 85 

. . r. . ■ 

jobs from 38 organizations were identified through less per- 
sonal means, includihg "cold call" contacts with the personnel 
directors of selected Fortune 500 industrial organizations, and 
with organizations who had recently advertised employment op- 
poftuhities ih a number of major newspapers , although it was 
hot possible to obtain the same degree of selection system 
documentation for this additional job sample. 

Next, open-ended interviews were conducted with the contact 
persons within the target organizations. Interviewees were 
encouraged to provide information about specific jobs for which 
selection procedures had been documented and to supply that 
docuraehtatioh in writing' wherever possihle. Included ih the 
information sought during this process was a specification of 
the kinds of knowledge, skills, abilities and other character^ 
istics sought, the types of selection devices that were used to 
identify them, factors relating to the development and imple- 
mentation of the selection system, -validity and reliability of 
the system in use, and the.humber of jobs affected by the 
process. Data were sought with special emphasis regarding the 
application of .the various sources of seler±±on data discussed 
earlier, including interviews, objective &sts, work samples, 
simulations and recommendations. As application blanks and 

resumes are used to gather a qualitatively vider variety of 

_ -_ _ ^^'^^^ _ 

selection data, respondents were asked rc indicate whether 
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jobs or education requirements were the important considera- 
tions. Additionally, resp'^ents were asked whether or not a 
license was required for h "1 a particular job. No data 

were available regarding se cf thought samples or detectors 

of deception in the preser i^mple^ 

Table 11 provides a summarized classification of the 239 
jobs for which selection system data were available. Approxi- 
mately 25% of the jobs in t'his table were drawn from "Fortune 
500" companies, 30% from other sizable businesses in the pri- 
vate sector, 25% from the federal and state service, education, 
and other public sector sources, and the remainder from miscel- 
laneous sources. Note that production jobs comprise the largest 
number of jobs ciassi£ied according to the taxonomy. It appears 
reasonable that tue majority of jobs that exist are those in 
which some amount of value is added directly by the employee to 
the 'product or service. It is also reasonable, that the greatest 
number of jobs on which selection systems have been documented 
are production jobs; since value added provides a more access- 
ible validation criterion than do other mearsur^s of performsmce 
effectiveness. By contrast, the fewest number of selection 
situations involved jobs classified as adaptive, since most 
jobs in this category are filled from within the ornanizatioTi 
through -promotion, or -from ^without through highiy^individual- 
ized means of selection which are beyond the scope of the 
present study. 
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TABI£ 11 



Taxpnon^- of Jobs Snrveyed by 

Level and Fanctxon: A Rexaresentative Summary 



Job Fmction 



Skill 
Level 



Unskilled 



Production 

assembly line 

worker 
cashier 
nonskilled 

clearical 
transportation 

worker 
waiter 

stock checker 
refinery worker 
miner 
cook 



Ifidntenance - 

£Lle clerk 
es^loyment 

rep, 
aide/orderly 
inspectica 

worker 
mechanical 

worker 
aSinistrative 
psychiatric 

attendant 



Managerial 

management 

txatnee 
store 

manager 
statistical 

manager 
administrative 

trainee 



Boundary 

duty collector 

checkout 

computer 

checkout 
salesperson 
human service 

worker 
toll collector 
library clerk 
revenue officer 
truck driver 



Maptive 



n = 32 



n = 11 



n = 8 



n = 18 



n = 0 



Skilled 



medical 

tetdjnician 
secretary 
engineering 

technician 
food processor 
mechanic 
c raft worker " 
' draffltsperson 
stafT nurse 
field worker 



con?>uter 

progrannaer 
administrative 

assistant 
firefi^rter 
airline 

maintenance 
alcohol 

coimiselor 
patrolmaq^ 
hospital 

corpsioan 
"diver 



foreman 
first line 

siroervisor 
buyer 

toll facility 

o^Sj^er 
sal^ manager 
correctional 

officer 



life insurance 

agent : 
manuf actxirer * s 

rep.' 
coi^Tiltant 
counselor 
product service 
assistant 

buyer 
claim?; 

authorize r 



junior 
consultant/ 
trainer 



46 



n - 22 



n = 22 



n = 16 ^ 



n = 1 



faculty ^member accountant 
senior technical biomedical 



speciaJ.ist 

>: architect 

Pro* . ^i^i^ 
*fessi^al eng^eer 

musician 

nuclear 
chemist 

choreographer^ 

surgeon 



31 
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technician 
process 

technician 
FCR manager 



38 



^manacpex of 
ma^oeting" 
manufacturing 



engineering 

manager 
product 

ST5>ervisor 
plant 

manager . 
police captain 



n.= 10' 



marketing 
manager 

consnercial loan 

officer 
lawyer 
technical 

s^ss 
purbhasing • 

agent 
foreign service 

officer 



politix:al 
^generalist 
management 



consultant 



4b 



lis 



n = 12 



46 



n * 5 



- - • ? - 

Many of these orgari±zations in our sampler particularly 
those with a long history of selection system documentation 
provided us with much information about selection practices that 
would not be reduced to statistical analysis. This section is 
therefore devoted to some of the more qualitative aspects of 
employee selection which may serve as an introduction to the 
qualitative presentation- that follows, "The concern of this 
section include the underlying motivation for choosing a given 
defvice, the issues and problems t&at arise with particular 
selection procedures, and' the changes organizations anticipate 
over the next several years. 

First, it should be noted that employing Organizations 
are highly reluctant to share data about their use of employee 
selection systems'. This was not unanticipated, due to the 
sensitivity and' proprietary nature of the information we were 
selling. However, evei after establishing our research creden- 
tials through references and correspondence and guaranteeing 
-conf identiality of the- data and their sources, nearly 25% of 
the organ±2Hl:ion cont acted did not wish, to contribute to the 
prsent study. During our study, we found companies particu- * 
lariy reluctant to share demographic data with us about the 



applicants. Those. who did seem to have data, and were willing 

to share it, fell into two basic classes: (1^ a group .of large 

i . _ _ _ _ 

organizations who were using biographical data explicitly as 



validated predictors, and therefore were willing to share 
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applicant characteristics with us; and (2) a group of people 
who were keeping applicant flow figures, and were in reasonably 
good posiitions with the EEOC* Otherwise, companies were par- 
ticularly defensive and reluctant to give out infarmatioh about 
the characteristics of people who went through the process. 
Their reticence was rooted in a fear that this data could be 
used against them. 

One of the emergent patterns in selection was the use of 
industrywide selection devices as in the petroleum and insur- 
ance industries. In both of these industries, several firms 
have pooled resources and looked at common needs and selection 
issues. As part of their cooperative rtudy, these groups ex- 
amined job analysis, test design and validation. In addition, 
the petroleum industry has been involved in cooperative studies 
on validity generalization in hopes of identifying job-related 
skills and appropriate tests which can be used by the companies 
within the industry. ~ 

When companies described the process of selection in which 
•they were engaged it was revealed -that everyone uses interviews. 
There is, however, great variability as to .whether interviews . 
are useSi as the first screen for an applicant, or whether they 
are usec3 for a final decision after . consideration of other kinds 
"of data such as applications or resumes. 



While all organi zations reported using intervTews, and Wod t 
organizations reported using skill tests for clerical people, . 
there are other consistencies which emerge in the use of methods 
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insurance industry while public braodcastihg companies rely 
heavily on resumes and trade publications which post job open- 
ings. These trends may be- reflective of similarities between 
jobs within a given industry and/or may reflect an indi?stry 
tradition in terms of selection methodology. 

Another trend that emerges in the use of testing as a 
•selection tool is the reduction in the use of personality and 
intelligence tests by most companies, usually for equal employ- 
. ment considerations. In one case where EEO legislation has had 
a visible impact oh the company's testing process, the firm has 
dropped testing from its selection procedures, instituting post- 
hire testing instead. After selection, a person is tested to 
see where he or she should 'be placed within the company- Those ^ 
tests that continue- to be used for selfection purposes are skill 
tests which generally have high face validity for the job. 
These tes^ts are given at lower levels for skilled and unskilled 
'jobs, clerical jobs, and hourly workers. 

The extent of structure given to the interview process 
varies across interviewers, departments .and organizations, in - 
our sample, two interviewers. are seldom required to use the ^ame' 
structure. While great variability exists between interviews 
for a single job, variability is even greater across jobs. The 
finance department might ^ interview people differently from the 
sales department. Another general finding in the use of 
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interviews is tjiat more interviews are recjuired for higher 
level jobs. A professional level ^applicant ''will probably be 
interviewed by many people whereas for lower level positions^ 
the interview process consists of one interview with someone in 
personnel. 

\ - ' - - - - - - 

When we asked, companies why they had instituted particular 

selection processes^ it was found that the motive behind the 
selection process w^s seldom a desire to understand or identify 
competence for, the job. Usually^ the selection procedure had 
been instituted because of a particular problem^ such as- turn- 
over ^ or because of equal employment issues. The companies 
seemed concerned about the need to protect themselves from any 
suit that might be brought against them> as well as wanting to 
show affirmative action. 

One set of ^issues of particular interest to us was how 
the* organizati^s identified the criteria for their selection 
process^ and what these criteria were .(i.e.^ specific knowl-. 
edge^ skills^, abilities, or other characteristics). We found 
two trends along this line, the first Being that, in our sample 
rigorous job analysis was clearly the exceptioli rather .than the 
rule. Very few companies had actually conducted job analyses. 
The secon^ trend was that the domain of job requirements tended 
to be identified by talking to the hiring supervisor and gener- 
ating a list of vpharacteristics the supervisor felt to be neces 
sary for the open job. One reason infrequent use of formal job 
analysis could be that organizations tend to think of job 
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analysis as appropriate for the development of paper and pencil 
tests, but not as an .important step in developing a method to 
evaluate resumes, application- blanks^' or as a prerequisite to 
the interview.* Since these latter techniques are more often 

_ i\ _ ... ... ----- _ - - - - _ _ _^ _ 

used than paper and pencil tests', job analysis is less common • 
The sample: vas askeS 'how the selection process was- originally 
developed. The practitioners, however, seldom had knowledge of 
the process development. Typically the process was described 
as evolving over a long period of time or else it was empirical # 
hot based on a concept of what was important to the job, but 
rather, on finding items which would empirically predict per- ^ 
formance. Organizations with empirically keyed selection tests 
(such as scored biographical data forms) must continually reval- 
idate them and therefore are more in touch with the development 

c 

of the process. Trait or skill measures are more likely to be 
considered logically related to job" performance and therefore 
receive less Questioning of the rationale for their use. 

The organizations mentioned three other ways their selection 
procedures were developed. 3y far the most typical was using 
published tests that already had established norms as well as 
documented reliability and validity data for similar jobs. In 

addition some companies spoke of developing their own inhouse 

■ ^. 

tests. The fewest number of organizations mentioned hiring out- 
side consultants to develop some of the more sophisticated kinds 
of selection techniques, such ai^ assessment centers for 
selection-- or coded interviews. 



When organizations were queried about the validity of their 
selection process, it was revealed that few organizations had 
conducted actual validity studies. Those that had done valida- 
tion primarily examined the validity of psy^chometr ic tests, 
especially those .used for selection into high-level positions i 
Also tests were more likely to be validated by the company if 
their relatedness to the job wasn't immediately clear. Person- 
ality tests with their low face validity frequently fell into 
this category. For lower level positions, tests in use were 
seldom validated! Instead the organizations relied on- published 
validity statistics. - ' 

Interviews were also .very seldom validated. Only a few 
companies were concerned with the validity of their interviews, 
because they didn't think of them as selection tests. The only 
other selection device which has been examined for validity is 
biographical data. , These 'data were usually validated for par- 
ticular jobs through keyed application blanks. Other methods, 
such as recommendations, staridard non-keyed application blanks ^ 
and resumes, tend not to be validated forms for collecting 
biodata. / 

When representatives of organizations were asked how their 
selection process had worked, they relied on anecdotes, fa'ce or 
content validity, -as evidence that their ongoing selection proc- 
ess was selecting the right people. They did hot rely on more 

» 

empirically rigorous methods for demonstrating the utility of 
their selection process. Where there was empirical evidence. 



that evidence tended to be gathered for the more complex, or 
less face valid procedures such as biographical data, person- 
ality tests, or assessment centers, 

A few organizations rejported that the problems they had 
encountered in the use of various selection processes were that 
ah^ihstrumeht was hot seen as being face valid, or that the 
selection procedure was not accepted within the, company itself, 
usually because of time c money constraints. 

One large 'trade organisation for the insurance industry 
also stressed that biographical data heed to be revalidated over 
time, since such data could not be counted on to maintain their 
validity over different years and different applicant popula- 
tions. This is a problem which is grossly overlooked by organi- 
zations using other kinds of selection methods. The biodata's 
instability over time is a severe problem for that kind of data. 
We 'found few orgahizat^ns outside of the insurance and petro- 
leum indu^st^ies which revalidated their selection procedures. 

Organizations involved in validity research stress misuse 
of selection procedures as a problem. They found it difficult 
to conduct adequate r^earch because operating groups often 
selected individuals on the basis of an unvalidated pilot 
instrument. Gost was. also a hindrance for some sophisticated 
procedures such as assessment centers. This was particularly 
true when large capital outlays were necessary, and techniques 
weiTe unfamiliar or hohtraditional. _ 

bur sample reported that data from selection procedures were 
usually maintained in company personnel files. Some companies 



kept data in regional of central files with the intention of 
doing later research. Many organizations were sceptical about 
inquiries around the use and maintenance of selection data 
files. We can only assume that this scepticism arose from EEO 
concern. Most organizations reported no further use of selec- 
tion data but those that did responded that the data were most 
often kept for one of three purposes: for maintaining conipli- 
ance records (EEOC and Affirmative Action) , for giving feedback 
to the applicants/ or for validation oh the instrument or 
selection techniques. 

Organizations were asked what changes they anticipated in 
their selection process. Several companies predicted a trend 
tbward increased job analysis / increased validation efforts (es- 
pecially for such devices as the interview) , and more behavioral 
methods of selection. Additionally/ one organization spoke of 
an anticipated need to justify selection and promotion decisions 
to the applicant. In the past this; had not been required but ^ 
the anticipation seems reasonable* 

When asked whether educational credentials were used as 
part of the selection process / those organizations that did use 
or consider educational credentials/ did so primarily for high- * 
skilled jobs. Organdzations seldom specified either the valid- 
ity of the educ^ational credential as a predictor of job compe- 
tence/ of the weight given to the information about education 
when making a selection decision. 
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Analysis of Trends and Practices 

The follcwihg data were noted for each of the 239^ jobs, 
outlined in Table 11, to permit a systematic quantitative 
analysis: the taxonomic job classification (job function and 
level of skill required), the sources, of data considered in 
making the selection decision, changes in use of each data 
source over time (e.g., whether the data source increased, 
remained constant, or decreased in us6) , and whether the data 
source had been validated with respect to the job by the hiring 
organization. 

Based on supplementary information and notes taken during 
the interviews the generic competencies required by the job 
were also recorded. These competencies fell into three broad 
groups: (1) intellectual competencies , including technical 
knowledge problem solving, planning and organizing, and ab- 
stract reasoning; (2) ihterpersbnal jcompetencies , including 
communication^ leadership, counseling, teaching skill and 
self-presentation; and (3) psychomotor competencies ,- includ- 
irxg manual dexterity, agility, "physical strength^ eye-hand 
CGordination. and speed and accuracy. Each job was coded by 
whether it required one or more of the competency group rather 
than whether an employing organization selected for these 
competencies. 

Table 12 presents a summary of the descriptive statistics 
regarding the use and validation of eleven sources of. selection 
data. The interview wr s, by^far, the most frequently used 



TABLE 12 

Use and Validation of Selection Data 



Source of Data 



Frequency Validated ^ percenta^je of frequency of use: 

of use use (Decreased Use / Constant Use / Increased Use) 



1. 


Interview 


73% ^ 


/ ^ 










2. 


Skill and Ability Test 


38% 


35% 


( 


5% 


92% 


3% 


3. 


Work History 


38% 


3% 


( 


0% 


100% 


0% 


4. 


Education 


31% 


5% 


( 


18% 


82% 


0% 


5. 


Biographical Data 


21% 


33% 


( 


8% 


89% 


^« 


6. 


Recommendation 


21% 


0% 


(. 


0% 


100% 


; 0% 


•7. 


Personality Test 


18% 


69% 


( 


15% 


83% 


2% 


8. 


'Bast Performance 


14% 


6% 


( 


0% 


100% 


0% 


9. 


Work Sample 


12% 


17% 


( 


0% 


94% 


6% 


10. 


License 


8% 


0% 


( 


0% 


100% 


0% 


U. 


. Simulation 


5% 


45% 


( 


8% 


84% 


8% 



) 
) 
) 
) 
) 
) 
)' 
) 
) 
) 
) 



Probability underestimated; see text 
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.source of selection information. The figure of 73% is in all 
likelihood a low estimate of the number of times both formal 
and informal interviews are conducted^ to screen employees, giv- 
en Scott's (1961) figure of 98% in his study of interviewing 
jpractices and the observation that many: of the respondents in 
our sample did not volunteer information about use of the cur- 
sory personal interview. Data from skill and ability tests and 
from work histories, are therefore probably considered less 
than half as often as the interview. Despite the interview's 
popularity, or perhaps because of it, the interview was among 
the least well-validated sources of information used in making 
a selection decision. Among the sources of selection data 
•listed, work history, educational background and recommenda- 
tions were also frequently used yet seldom validated. The most 
widely validated source of information was the personality 
test, due mainly to the fact that most employers relied on 
published validation statistics to justify their use. Skill 
and ability tests and biographical data were two additional 
sources of information . that were frequently used and relative- 
ly well validated in the field. The performance simulation, 
largely because of its complexity and the expense involved in 
its administration,* received significant field validation, 
despite its infrequent use. 

The remaining data in Table 12 are devoted to trends in the 
use*of the various data sources. As a rule, the use of data 
sources has remained substantially unchanged for each selection' 



-1.116-127 



system. Nevertheless, conclusions may be drawn from tife pro- 
position of instances in which the use of a particular data 
source was increased or decreased over the life of a selection 
program. Specifically, the educational requirement and the 
personality test show trends toward decreasing use, undoubtedly 
reflecting recent court decisions striking down educational 
requirements as unnecessarily discriminatory and personality 
testing as not demonstrably relevant to job requirements. In 
the absence of trends toward the increased use of other data 
sources, one can conclude (1) that the selection interview, due 
to its mere pervasiveness, is being counted upon more and more 
heavily in making the final selection decision, even though it 
is among the least reliable and valid of the selection 
techniques; and (2) that the use of credentials and personality 
test, though viewed as important sources of selection data, is 
simply being discontinued for fear of reprisal in the courts. 

Somewhat surprisingly, few differences in the use of pax- 
ticular data sources were found between major job functions. 
The data in Table 12, therefore, present a largely accurate 
picture of the use of data sources for production, maintenance, 
managerial and boujidary jobs. A few differences by job func- 
tion, however, were obtained at statistical levels of signifi- 
cance (p < .01^ . Skill and ability testing was nearly twice as 
prevalent among maintenance jobs than among others, and educa- 
tion and simulation data were considered twice as often for 
managerial, jobs than for production, maintenance or Boundary ^ 
jobs. 



Table 13 illustrates the characteristics of the job sample 
in terms of the competencies required by the job according to 
job function and skill level.* In the analysis of competen- 
cies by job function^ the majority of managerial and adaptive 
jobs in the sample require some specified level of intellectual 
competence; the majority of managerial, boundary and adaptive 
jobs were found to require interpersonal competence; and pro- 
duction and maintenance jobs largely require psychomotor com- 
petence. The second part of Table 13 shows the data collapsed- 
across job functions and examines required competency broken 
down by the skill level specified by the job requirement. , 
The great majority of unskilled jobs require psychomotor 
competencies to the relative exclusion of intellectual- 
interpersonal competencies, while jobs designated as skilled 
lean toward the requirement of psychomotor competencies but 
are, in the main, more balanced in their requirements of the 
three generic competency groups. The professional skill level 
jobs in the sample require a much higher degree of intellectual 
competencies than interpersonal or psychomotor competencies, 
although these last two competency groups are significantly 
represented. . Considering all jobs in the sample together, the 



* Note that the percentages listed which^ are percentages of 
the row total exceed 100%, since many jobs are represented 
under more than one competency. 
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TABLE 13 



1 



Job Sanction 

s . ' 

Production 

Maintenance 

Managerial 

Boundary 

Adaptive 



Characteristics of Job Sanipier Breakdown by 
Job Type cmd Scill Level by ConDetencies Reqsxred by the Job- 

CoQ^tency Seqmxed by Job 

n intellectual ' Interpersonal 



(109) 
(38) 
(40) 
(46) 
(6) 



36% 
34% 
65% 
28% 
100% 



9% 
24% 

85% 
78% 
83% 



Psychomotor 

92% 
68% 

0% 

3% 

17% 



All Jobs 



(239) 



41% 



39% 



59% 



Cosqpetency Required by Job 



Skill Level 



Intellectual 



Interpersonal 



Psychomotor 



Unskilled (69) 

* 

Skilled (107) 
Professional (63) 



10% 
34% 
86% 



26% 
46% 
43% 



74% 
58% 
44% 



All Jobs 



(239) 



41% 



39% 



59% 



Percentages are based on proportion of row totals 
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competency representation, although favoring psychomotor 
competencies to some degree becauise of the large number of 
production jobs surveyed, appears reasonably balanced. 

with these descriptive statistics as background, trends in 
the usie of particular selection data were examined by the level 
of skill and the competencies required by each job. Table 14 
presents these data for the eleven data sources of the previous^ 
table* A consistent finding is that the frequency of use of 
selection data varies directly with the level of skill required 
ir. the job. This finding is consistent for the interview, work 
history, education, recommendation, past performance and li- 
cense. Skill and ability tests show the only significant 
departure froiS this trend: employers who seek professional 
level skills -tend to shun objective testing in general as a 
mechod for ascertaining competency, while significant numbers 
of employers who are hiring unskilled and skilled employees may 
not be able to assume the presence of minimal skills or abili- 
ties in applicants and therefore find skill and ability testing 
to be useful additions to a selection system. - However, with 
all sources of selection data talcen together, the median number 
of data sources considered across jobs is 1.3 for unskilled 
jobs, 1.8 for skilled jobs, and 3.3 for professional jobs, a 
highly significant l^ear trend in the use of selection devices 
with particular emphasis in professional-level jobs. 

The final tables in this section illustrate the likelihood 
with which sources of selection data are used for jobs reguir- 
ihg intellectual, interpersonal, or psychomotor competencies. As 



TABLE 14 

Frequency of Selection Data Use Related to 
Level of Skill Required in Job 



Source of Data 



Unskilled 



1. 


Interview 


65% 


2. 


Skill and Ability Test 


36% 


3. 


Work History 


22% 


4. 


Education 




5. 


Bipgraphiccil Data 


28% 


6. 


Keccosaendation 


7% 


.7. 


Personality Test 


17% 


8- 


Past ^rformance 


6% 


" 9. 


Work ^ample 


.10% 




/ 




10. 


License 


3% 


11. 


Simulation 


4% 


Number of Ccises: 


69 









LEVEL OF SKILL 
Skilled 



70% 
50% 
12% 
22% 
20% 
2^ 
17% 
8% 
10% 
6% 
5% 
107 



Significant 
Professional Trend 



86% 
13% 
67% 
49% 
18% 
36% 
14% 
35% 
16% 
19% 
5% 
63 



linear (p < .01) 
nonlinear (p < .001) 
linear (p. < .001) 
linear (p < .001) 

lineax: (p < •001) 

linear (p < .001) 



linear (p < .001) 
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the use of selection devices was shown to be related to skill 



level, "which was itself correlated with type of competency re- 
quired by the job (see Table 13) , the f igure~s~in these tables 
were adjusted for the correlation between level of skill and 
competency requirements* Table 15 shows the' relative degree-^ 
to which a source of data will be employed as a function of . 
whether or hot a particular cos^etency is required by the job* 
It is evident here that work history, education and simulations 
ar'e' more likely to be considered when intellectual skills are 
required than when they are not; skill and ability tests are 
the only (3evic:es „ tfiat are by themselves more likely to be used 
when interpersonal skills are required; and skill and ability 
tests and licenses are more likely to be considered when psycho- 
motor competencies are at issue. 

In most cases however , it is likely that more than one 
source of selection data will, be used, so it is necessary to 
examine which data sources tend to bemused together most often - 
in selecting for certain competencies. To this end a discrim- 
inant analysis was performed for each of the 11 sources of data ^ 
considered by whether of hot a specific competency was required 
for the job:, and Table 16 presents ±he -results. Taken together, 
education a^id simulation data are more likely than chance to be 
considered in selecting for intellectual skills while, curious- 
lyr skill and ability tests are ignored to a degree greater 
than would be expected by chance. Simulations, personality 
tests and recommendations were found to be used more often when 
interpersonal competencies were required by the job, while skill 
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. - -^TABLE 15 ' 

Frequency of Selection Data Use Related 
To Con5)etency Requirement of the Job^ 



Is Cofls^eten^ Required? 



Intellectual. 
NoAes 



Interpersonal 
NoAes 



Psychomotor 
No/Yes 



^- 1 . Interview 

2. Skill and Ability Test 

3. Work History 

4. Education 

i _ ..... 

5* Biographical Data 

Recommendation 

- 7. Personality Test 

_ 8. Past Performance 

-9. Work Sample 

10 • License 

11. Simulation 



71%/77% 
48%/20%** 
30%/53% 
l7%/46%** 
20%/22% 
17V29% 
17%/15% 
12%/20% 
ll%/i4% 
8%/10% 
1%/10%** 



76%/71% 



25%/43%**. 
37%/40% 
29%/28% 
22%/21% 
i7%/27% 
13%/22% 
12%/18% 
13%/10% 
12%/3%* 
1%/10%** 



73%/73% 

28%/42%* 
,39%/38% 

40%/21%** 

27%/17% 

22%/2l% 

25%/i0%** 

17%/13% 
8%/15% 
2%/13%** 

10%/1%.** , 



*p < .05 
**p < .01 



corrected for level of skill required 



\ 



ERIC 



.1.123^^3^ 



TABLE 16 

Selection Data Considered as a FTsnc1:ion of Ccn5>etency 
Required by the- ^rb 



Competency 
Required 



Data Sources Most 
Likely to Be Used 



Data Sources Least. 
Likely to Be Used 



Intellectual 



Educatipn 
Simulation 



.27** 
' .17* 



Skill and Ability Test .--.23% 



Interpersonal 



Simulation 
Personality Test 
Recammendation 



.19** 

• 11 

• 15* 



Skill and Ability; Test -.20^ 
License -.12 



P^chomotor • 



License .19 
Skill and Ability Tfest .14* 
Work Sample .11 



.Simulation 
Personality T^st 
Education 



-.23** 
-.20** 
-.21** 



^Correlations aure with requirement *of skill (0 
-corrected for skill level of applicauxt^ 

*J) < .05 

**p < .01 . 



= not required by jd^/ 1 = required by job) , 
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and ability tests and licenses were considered less often. 
Finally, when psychomotor competencies are required^ licenses, 
skill and ability tests and work samples tend to be used indi- 
vidually or in combination in making a selection decision, while 
simulations/ personality tests and educational background are 
the least likely sources of data to be considered. It is in- 
teresting to note that many of the data sources which are like- 
ly to be used in selecting for intellectual and interpersonal 
competencies are the least likely to be used in selecting for 
psychomotor competencies/ while those sources of data used in 
selecting for psychomotor competencies, are correspondingly less 
likely to be used in selecting for either intellectual or inter- 
personal competencies. 

Implications for Educators and Employers 

It seems reasonable to expect that the selection devices 
used by ah employer to make a hiring decision would reflect in 
some way the job competencies required of the applicant. In 
practice/ the degree to which this is true depends on how* broad- 
ly the job competencies are defined. If one is concerned with 
selecting for specific skiiiS/ abilities or performer charac- 
teristics such as manual dexterity/ negotiating skill/ the 
ability to delegate taskS/ knowledge of the latest health care 
technology, or planning skill/ the state of the practice in \ 
selection is woefully inadequate. The primary reason for 

-1.125- ' 



this is not the often low reliability or validity of individual 
selection devices, but rather the general absence of job 
alyses on the basis of which selection devices could be devel- 
oped* ^ Selection procedures are chosen without much evidence 
that they tap a relevant domain of jpb-related skills. With 
the exception of training programs with practitioner-teachers 
(for example, in the health care professions) , there is very 

little contact between the world of education and the world of 

/T __ _ _ _ . 

work/ That will help educators do a better job preparing 

students for life outside the classroom. 

In reality the selection procedures used by most employing 
organizations provide little insight into the specific know- 
ledges, skills, abilities and characteristics required for 
work. Without job analyses to inform selection systems, it 
is surprising that any validity at all can be obtained by most 
selection procedures currently in use. If measures of compe- 
tency are undertaken at a more general level for selection, 
however, some reassuring observations can be made. The present 
empirical analysis of. the state of . the practice was focused 
at generic competencies — intellec^'ual, interpersonal and 
psychomotor- — then specific competencies within each of these 
categories. With face validity being a prerequisite for the 
use of selection techniques and the nature of inferences drawn 
from the data collected through the use of these techniques, 
current selection practices are more likely to detect compe- 
tencies at the generic level which are related to job 

• 137 
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requirements* For example, most jobs classified as boundary 
positions emphasize interpersonal competencies (see Table .:13),r 

and the selection interview, a device which is sensitive to 

/ . ■ _ ' . . _ . . . . . . . . _ ■ 

general interpersonal skill, is used in the majority of employ- 

inent situations requiring interpersonal skill (disregarding for 
the moment that the ubiquitous interview technique is used to 
select for nearly all jobs) • Therefore, applicants for gobs 
which serve a boundary function have a high chance of being 
assessed on at least one relevant job dimension (interpersonal 
skill) giveir^urYent selecl:ion practices, even, in the absence 
of a thorough job analysis. Table 17 illustrates specific com- 
petencies grouped by generic competency catego.ry and measured 
by the sources of data considered in the previous sections of 
this chapter 

However, simply because a 'selection device is. particularly 
sensitive to certain generic competency dimensions, that does 
not necessarily imply that information related to relevant 
dimensions will be extlacted by the employer in making a final 
selection decision. One can only estimate the maximum availa- 
bility of relevant selection data by comparing generic compe- 
tencies required for a job and the use of a particular selec- ^ 
tion procedure which has a disposition to focus oh these 
competencies. " The data collected on the 239 jobs allowed such 
a comparison to be made. The generic competency measured by 
each selection device (See Table 17) was .compared with the 



TABLE 17 



Generic Claissif ication of Tj^ical Coa^tencies CSbtained Through 
^ployee Selection Data Sources 



Generic Coppetency: 
Data Source: 
Interview 



Slcill and Ability Tests 



Personality Tests 



Biographical Data_ 



ra 



. Es^rience 



ra 



Educational Level, 



ra 



Variafaies- Tapped J^iv^cH Generic Competency Area 
Intellectual Interp- t> rsonal Psvcdiomotor 



General intelli- 
gence 



Verbal compre- 
hension 
Verbal abili^ 
Himerical centre- 

hension 
Humerical abili^ 
General reasoziing 
Creativity and 
judgment • 
Spatial realtions 

Intellectual 
dysfunction 



Educational level 
Degree status 
Grades (GPA) 
Interests 



Sociabili^ 
Interactions with 

others 
Communication 
-ability 



Sociability 

bominaxvce . 

Sensitivity 

Independence 

Extraversion 

Cooperativeness 

Social network 



Job proficiency Job proficiency 

Area of speciali- . 

zation 
Years of stu^ 
Grades 

Academic honors 



Manual dexteri^ 
Visual acui*^ 
Auditory acuity 
Finger d^xerity 
S speed 



Job proficiency 



(Table 17, continugtd) 

Generic Competency: 
Data. Source: 



Variables in Each Generic COTpetency Area 
Intellectual jg^erperscaial Psychomotor 



License. 



ra 



Reconaoendation 



Content mastery 
Recognition of 

principles 
Application of 
knowledge to 
^rbblem solving 

Job proficiency 



Past Pcarformance 

Work Saii5)les/Siniulations 



Job proficient 

Planning and^ 
organising 
Decision making 
Creativity 
initiative 
Judgment 
Thoroughness 
Abili^ to learn 
Analytical slcills. 



r « accessed through resume 

a ^ accessed through, application 



Responsibility 
Cooperation 
Independence 
Sociabili^ 

Job proficiency 

Comxmication 
Leadership 
Delegation ~ 
Enpathy 

List^ing slcill 
TeasBHrbrk 
Human relations 
Teaching and 
sx^rvisihg 



Ability to inani-_ 
pulate equipment 
for acconglish* 
ment of task 



Job proficiency 



Job proficiency 

Coordination 
Strength 
Steadiness 
Positioning 
Seeing : 



Balance 
Speed 
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required generic competencies of^ each job to derive an indi- 
cation of how likely- selection systems were to produce data on 
required intellectual , interpersonal or psychomotor compe- 
tencies (whether or not these data were' ultimately used in 
making a selection decision). For 95% of the cases in which 
either intellectual or interpersonal skills are required by the 
joby selectj.on procedures were used which were appropriate 
sources of data on these competencies. In only 80% of the 
cases where psychomotor skills were required, however , did the 
selection procedures used have the potential to provide useable 
data in this competency area., due primarily to the fact that 
the interview, an inappropriate source of data on psychomotor 
competencies, is used in many of these jobs in preference to 
more direct measures of psychomotor skills. 

This finding raises the issue of using selection devices 
which have a high probability of yielding datr. that are inap- 
propriate to the requirements of the job. In reference to the 
previous example involving selection for psychomotor competen- 
cies, it is clear that reliance on the interview as the sole 
source of selection information may tempt an employer to base a 
selection for a job requiring psychomot^ skills on' information 
related to intellectual or interpersonal competencies which 
have either a tangential or a non-existent relationship to the 
job requirements. Inde'fedv in ^over 70% of the cases in which 
psychomotor skills were not required by the job, selection de- 
vices were used which had at least the potential of providing 



information on such skills. Similarly, in 90% of the cases 
where intellectual competencies aiid interpersonal competencies 
were not required, selection devices were employed which pro- 
vide data on these competencies. These percentages are probably 
inflated as representations of the actual use of inappropriate 
data in making a selection decision, but nevertheless they 
illustrate present dangers of such decision-making processes. 
As it is seldom clear even to employers what competencies are 
required by a job, further errors introduced by the application 

~of inBpproE>riate sefeQtion criteria make it difficult to pre- 
dict whether an otherwise competent ' individual will pass the 
selection hurdle. 

In sununary, th.e major findings of the present empirical 

'study of data sources considered in making a ' selection deci- 
sion are as follows: ^ , 

(1) With the exception of the personality test, the major- 
ity of selection devices and data sources have not been v^idely 
validated by employing organizations that use them. The 
selection interview in particular, though it is used by over 
90% of employers, is not only among the least reliable and 
valid selection devices in existence but is also among the 
least validated of data sources within employing organizations. 

r 

(2) Differences in the use of selection devices do not 
vary with job functions themsfelves^ but vary with the type of 
competency required by the job and with the level of skill 
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expected of the employee. at the point of selection. The empha- 
sis oh the use of different selection data sources # as a gen- 
eral ruler increases with the level of skill required in the 
jobr independent of the types of competencies required. 

C3) General trends in the decreasing use of education and 
personality testing for employ nent selections^ reflecting court 
decisions on cases involving employment practices and problems 
arising from attempts to comply with EEOC guidelines are not 
balanced either by trends in selection system validation or the 
increasing use of other selection devices. This suggests; both 
an increasing reliance on other/ perhaps less reliable or valid 

information sources, and a lessened; emphasis on entry selections 

■ * • 

in general.. 

(4) Though in many instances the techniques which are used 
to select for particular competencies are appropriate to the 
task/ there are significant numbers of instances in' which 
inappropriate sources of data are used to select for certain - 
competencies while other # more appropriate measures are ig- 
nored. This appears to be due to both a general lack of good 
job analysis among the jobs we surveyed and the overuse of data 
services which are interpreted beyond their limitations. 
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'JV. CONCLUSIONS 

Discrepancy Between ideal and Actual Practice 

Tfiroughout the preceding pages we have described, analyzed, 
and criticized both the research and the practice in competence 
assessment for selection. The practice of selection has been 
critiqued by comparison with the research oh selection methods 
and by comparison with more generalized prescriptions for im- 
proving employment selection. Research on a variety of methods 
has likewise been compared favorably with the ultimate goals of 
near-perfect reliability and high validity. While the individ- 
ual criticisms and comparisons throughout the text stand on 
their own, this is an appropriate point to attempt to tie them 
together and make a statement about the discrepancies between 
ideal selection through systematic competence assessment, and 
the reality of selection practices in functioning organizations. 

As early as 1913, Muhstefberg was studyirig street car 
mbtormen and prescribing methods for improving selection for 
that job. At about that same time^ Otis was doing the basic 
research oh paper and pencil selection tests which he later 
turned over to the Army for the development of the famous 
"Alpha" and "Beta" tests used to select World War I soldiers- 
Obviously, selection testing is hot something new. For more 
than 65 years employers been trying to 4eyelop^'devices to 
identify people who will be^cQmpetent workers in the organiz-' 
ation.. Since the very early days of selection testing. 



certain practices have been prescribed for- optimizing tire 
validity and" utility of selection techniques. It is both \ 
heartening (in terms of foresight) and distressing to notice 
how appropriate those prescriptions are to this day. 

In 1923 Freyd published a series of three journal articles 
in which he outlined in detail the principles and practices 
of "Measurement in Vocational Selection." He argued that, an 
organization should identify that department in" which the 
g r ea tes t s a v ings could be real4z-ed frc^m improved selection^ 
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The actual design of a selection instrument should then follow 
10 steps. 

1. Do a job analysis to identify what leads to success or 
failure on the job. 

2. Identify a single measure of the criterion of success. 

3. Select a sample. Inexperienced applicants are pre- 
ferred to incumbents. Identify and study age or sex 
differences. ' 

4. Develop an exhaustive list of ^abilities required for 
Jthe job and reco'mmend procedures for evaluating each 
one. 

5. Find or devise appropriate measuring instruments 
(not restricted to tests) . 

6. ^ Admihis^ter the tests and mieasures under carefully 

controlled conditions.. 
J. Statistically compare testing results with the-, 
criterion. 

S. ComlDine multiple measures for maximum correlation. 
' _ ^1.134^^-^ 



9^ Justify the measuring instruments by comparing their pre- 
dictive accuracy with selection methods already in use. 
10* Only if the comparison is favorable,, should the new 
measures be ihstalledf being sure they are properly 
usedi Continuously monitor their predictive accuracy and 
adjust them "to the type of applicant and changes in 
industrial demands^ (Freyd (1923) as cited in Guion, 
1965.) / ' ■ 

As note^ by Guion (1976) these "tenets of orthodoxy" are 
remarkably contemporary. Throughout this chapter we have made 
the exact same recommendations. We have criticized many cur- 
rent methods of selection because they don't specifically ideh^ 
tify and tap job skills derived from job analyses (1) . The 
criterion for employee selection is often unclear (2). Where 
criterion related validation is attempted, it is often of the 
concurrent variety, i.e., incumbents are measured against their 
current performance (3) . Selection techniques often tap only ,one 
skill or a small subsample o^ the skills needed for the job 
(4,5).. Selection procedures such as the interview are hot con- 
sistently applied to all candidates (6).' Statistical valida- 
tion is avoided in favor of conceptual content validation (7) . 
Decisions are made on the basis of a single piece of selection 
data such as one interview or ah aptitude test (8). Cost bene- 
fit analyses are foregone (9) and. new methods are guicJc-iy 'ini^ 
plemented (id).. Freyd 's caveats have gone unheeded to a large 
(extent. 
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While not aii selection instruments have been as" rigorously 
validated in research as Preyd would hope, " academic research in 
this area exceeds operational research in volume and quality* . 
•Job Analysis, rigorous test design and criterion related vali- 
dation, however, have remained relatively uncommon in practice* 
This discrepancy between the practice and the research is 
discouraging. The methodology for daing all three of these 
things is well documented, All that is really needed is time 

and money to implement methods--aii:'ea^~proveTT^by^esear^ The 

...... ' % 

impact of new research on selection practices would be. marginal 
compared with that of the employment^ practice catching up with 
the current state of the art. ^ ' 

In his 1976 review of selection practices for the Handbook! 
of Industrial/Organizational Psychology (Ed. Dunnette) , Guion 
•contends thi^t the aforementioned tenets of orthodoxy are still 
contemporary as guides to good seiectionL:^ractice but that f 
they are often overlooked. He concludes that the 1970 EEOC ' 
guidelines and court rulings in support of validation would >^ 
bring more companies back to the rigorous selection practices 
prescribed in 1923 and JLoday. _ While this seems ta^have been a 
logicaT'^prediction, and clearly the intent-of the EEOG and 
Congress, our current data do not bear -witness to si^h a trend. 

Factors Underlying the Discrepancy 

Considering the ever-growing body of research in employee 
selection technology, why has there been so little progress in 



the, State of the practice? Though typical applications of. the 
selection procedures discussed yield poor reliability aiid.vali-* 
dity,- carefully managed studies involving job ^analyses and per- 
formance criterion validation have shown that many of these pro 
cedures could be of high utility in employee selection. Prom a 
conipetency based perspective, "predictive validation" is the key 
to improved selection processes; that is, the data obtained 
through a selection procedure should be related in some way to 



"the~"competencies required for. performance of the job, A demon- 
strutted criterion relationship is important .for choosing a se- 
lection process that has utility in prediction (over chance 
levels of accuracy) and is cost-effective.' Why, then, are 
there so few validation studies undertaken to demonstrate these 
relationships? 

The use of selection procedures satisfies an organization's 
need to make a "careful decision". I-n their own eyes selection 
data must first have a lace-valid relationship with performer 
qualities that seem to be^impdrtant on the j6b« In most casesr 
this satisfies the organization; studies are seldom undertaken 
to test the criterion-related validity of face-valid tests that 
reflect employers' assumtiohs about job-related character- 

istics. In addition, the uniform guidelines, despite their 

\ . ^' X. , ■ • - , ■ 

^intention to jtgply fair and consistent standards for the 

evaluation and use of selection devices, have made selection 

testing less appealing to employers who feel that data used 

in making a hiring decision can be ~used against them in a 

discrimihatioh suit. Also affecting the choice of whether or 
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not to validate is that most employing organizations are simply 
too small in terms of jobs and people to make test validation 
feasible,' Either the costs of such at process' a^e prohibitive 
or there are €00 few people in- a given job to-make any valid- 
ation statistics meaningf ul?< Since many organizations either 
ignore the state of the art and remain satisfied with face- 
validity, see the^ EEOC guidelines as a threat, or are simply 
too small to afford validation^, it is a wonder that validity 
studies are carried outsat all, . . ■ ' 

All these trends, however, are counter to the -intuitive 
notion that good, valid selection should lead to greater organ- 
izational performance and productivity. Why organizations find 
the trade-offs too great in pursuing this goal seems a mys- 
tery. Theirs is, in essence, a reactive mo4e which suggests 
why the state of the practice has riot improved over the past^SO 
years. Despite these factors, a number of or^aihizatiohs in our 
sample have persisted in attempting to validate their selection 

precedures. Data on those organizations suggest a model for^ 

___ J-.- __. 
explaining differences in the nature of hiring organizations 

<k _ • 

thenselves which, in. combination with internal and, external 

pressures oh the organization, largely determine whether or not 
validation oi selection procedures will be pursued. Likewise, 
it will be proposed here that the lifecycle of" ah organization 
has a strohg impact on the state of the selection and valid- 
ation "strategies used in that organization. 

Wright (1969) # Rothschild (1976), and others have related 
business plahnihg to certain stages .of organization 
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maturity and strategy. ' These stages largely characterize the 
behavior of organization with respect to the role of mahage- 
raehtf strategic planning^ policies, procedures, and information 
and measurement systems. Organizations may be characterized by 
four stages of maturity: 

1* The embry onic organiz afcion is characterized by small size 
and initial rapid growth, technological change, and pursuit of 
customers for its products and services. Leadership in such an 
organization tends to take an entrepreneurial, seat-of-the- 
pants approach, and such an organization is therefore highly 
dependent on the environment for its continued support. 

2. A growth organization is still growing rapidly but tends to 
be larger, better known and established and in a product or 
service area where new entry by other organizations is more 
difficult. Management, like- the organization, remains market- " 
oriented and is often torn between fulfilling boundary arid 
managerial roles. . 

3. A mature organization is still growing albeit at a less 
rapid rate; holding and defending the products and services it 
has el^tablished is a primary concern. Here the managerial role 
beconiefe more administrative and the organization adopts a goal 
of stability and maintenance of market. share. 

4. An aging organzation is faced with a lessening of demand, 
competition, and diversity of products and services. Manage- 
ment becomes opportunistic and relatively short-sighted. The 
goals of the organization are either to harvest the product for 
ail it is worth, or to divest. 
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The proposition here is that the strategy to validate the 
use of employee Selection instruments is bound together with 
the emerging organizational character defined by its level of 
maturity* A second factor affecting the likelihood that an 
organization will or will not undertake selection validation 
involves sophistication about testing and selection dthroughout 
the corporation and in the person of the personnel adminis- 
trators who cheese and use selection methods. Sophistication 
in the use and design of selection instruments is likely to 
increase as an organization passes from the embryonic to the 
growth and then to the mature stage* By sophistication we mean 
the insight gained from experience with successful and unsuc- 
cessful selection attempts / as well as formal knowledge of the 
difficulties involved in defining selection criteria, con- 
structing predictors, analyzing predictions, and refining the 
usv^ of the selection device for making equitable job-related 
employment decisions. Since the personnel function evolves and 
is elaborated over time as an organization grows (Katz & Kahn, 
(1978)^sophistication in use of selection procedures should 
grow as the personnel function grows and the personnel staff 
becomes more-' specialized. 

If you consider for a minute the orientations of organiz- 
ations at' different stages, it is clear that personnel func- 
tions have an increasing potential over time to help an organ^ 
ization maintain a stable, effective work force as the or- 
ganization approaches higher levels of maturity. When an 



organization is aging, however/ the personnel function may 
atrophy because ther^ is c. decreased need for having, and no 
long-term commitment to having, the best work force in the face 
of less competition or decreased market share: Nonetheless, 
regardless of the stage an organization has reached in its 
lifecycle, more sophisticated corporate decision makers and 
personnel staff will be informed and presumably more rigorous 
in their efforts at design and evaluation of selection 
procedures. 

Factors outside the organization also come into play in 
determining what strategy will be used for selection. One 
factor that stands out as an important determinant 6f^ the 
ultimate selection strategy is whether or not an brganization 
is monitored by the Federal government or the courts with re- 
gard to its past and pres<?nt selection practices. Monitoring 
may come about through past violations of Title Vii which have 
surfaced in the courts, or by virtue of the organizatioTi^s 
being heavily regulated by government agencies. 

Our observations suggest that regardless of the stage o£ 
the organization's maturity and selection and testing 
sophistication, if the organization is vulnerable to being 
monitored, the strategy it will adopt is to document its 
selection practices rather than to validate, its procedures or 
to continue with undocumented past practices. JLf one accepts 
that the first goal of an organization is survivai"^ and the 
second goal is growth, then it is unlikely that public scrutiny 



of the organization's selection and hiring practices will be 
countered by amassing voluminous statistics documenting .that ho 
adverse impact occurs {i.e., hiring to conform to the 4/5 rule 
of the 1978 unif orm.gaidelines on selections) , but which do hot 
necessarily document the validity of the selection criteria. 

This trail of paper is essentially a protective measure • Even 

__ ... 

if validation is to be undertaken ultimately documentation of 
-present practice, applicant flow, and impact ..is a hecessary 
protective firsts step. 

Figure 1 shows the logical relationships between stages ,of 
organization maturity, external factors, seiectioh ahd testing 
sophistication/ ahd the ultimate selection strategy adopted by 
the organization. / ; 

In light of our sample survey and of the factors we have 
identified above, this figure presents (in the form of Venn 
diagrams) our basic -hypotheses about (1) why and when an 
organization devotes to a selection (hiring) process is a . 
function of three sets of variables: 

1. Stage of organizational development (the life cycle) ; 

2. - Visibility of "the^-organl-zat-ron to previously excluded 

groups f cohsumerSf monitoring agencies; and 

3. Personnel and corporate sophistication about testing and 
selection. 

At a very general level of analysis which will allow us to 
compare corporate resources of various types on one scale, we 
have identified four different levels of resource allocation to 



Figure ■ 1 



Combinations of Factors Determining 
Resoxirces Devoted to Organizational Selection Practices 



The interaction of circulcir areas depicting stage and the other factors represents 
the resoTirces deyoted to selection. 

In each case the stage of organizational development of life cycle is given. 



CASE 1: 




When an organization is neither visible nor sophisticated in selection 
practices there is no- intersection and no extraordinary resources are 
devoted- to selection. 
SELECTION IS AD HOC. 




When an organization is visible but not sophisticated in selection practices, 
resources will be put into the short range protective stategy of record 
keeping. 

SELECTION IS DOCUMEIOTED. 




When an organization is sophisticated in selection practices, but not visible, 
resources will be put into efforts to improve the accuracy of predictions of 
methods. 



SELECTION IS VALIDATED. 
CASE 4: 




When an organizaion is both visible and sophisticated it will be under short 
..term pressure to protect itself and long term pressure to improve selection 
practices. 

SELECTION IS DOCUMENTED AND VALIDATED. 

Some resource expenditures will benefit both efforts but overeill resources 
required will increase. ^ - 
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selection practices. They include: (a) Ad hoc selection which 
requires no extraordinary resources to function; (b) Documen- 
tation which requires timer money and- manpower to record and 
maintain demographic data on applicant flow; (c) Validation 
which requires time, money/ manpower/ and a research and 
'development commitment; (d) Documentation and Validation which 
requires a greater resource pool to accomplish both. Figure 1 
suggests which level 'of resource allocation will follow from a 
given combination of, the three sets of factors we have 
identified. 

In addition to using the three sets of factors to identify 
what level of resource expenditure can be- expected in a given 
firiflr we^can hypothesize how visibility^ and sophistication will 
be distributed across stages of deveiopmerit. Given that dis- 
tribution we can also predict during which- stages the goal of 
validating .selection measures of competence is likely to be 
attained. Thus, on the basis of what we observed in our sample 
survey we propose: 

1. Visability is -low for hewr young/ embryonic organiz- 
ations which have yet to establish themselves in the public 
eye or grow to proportions qualifying them for monitoring. 
Otherwise/ visibility is constant with a slight decline possi- 
ble among aging firms that have survived previous monitoring. 

.2. Sophistication increases as an organization develops 
(see Katz & Kahn/ 1964)/ evolving a personnel function and 



expanding its influence. This function and its forces atrophy 
in aging organizations with* decreased hiring and less active 
competition. .. _ 

3. Validation (or the documentation/validation comfcinration) 
is^ likely to occur in mature organizations most often/ followed 
by growth organizations,, and is least likely, in early or late 
stages of an orgar*-ization' s life cycle. 

Figure 2 reflects the relative likelihoods of these con- 
ditions. The axes on the probabrlity distributions are not 
scaled because at this point the graphs are only-suggestive. 
These predictions reflect findings from the present state of the 
practice survey. In general, industries on the decline, (for 
example, supermarkets and railroads ^coinpanies-)—reduc€="their use 
of formal assessment procediires, while organizations which are 
both on the increase and have a vested interest in maintaining 
future stability (for example oil and^ other energy-related 
industries, and insurance companies) show an increase in selec- 
tion recearch. But even when an organization is mature, grow- 
ing, and seeking stability, it still must be relatively free of 
external scrutiny or must also possess ajtnan 

sophistication in selection procedures which can, produce a long- 
range manpower strategy to support a validation effort. Other- 
wise the tendency will be either to document past practices or 
to continue selecting oh an ad hoc basis. 

This model raises interesting implications for a prescrip- 
tion to change the state of the practice in employee selection. 
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As was originally envisioned^ the* EEOC guidelines were intended 
to have a facilitating effect on ensuring validation of selec- 
tion procedures while maintaining fairness in hiring. Recent 
court decisiohSf however, have resulted in employers reducing 
their reliance on objective testing for selection, turning 
instead to less rigorous sources of data such as the interview 
and retyping to content validity as their accepted standard for 
selection practices. - The model suggests that no amount of 
Federal policy regarding selection practice will encourage val- 
idation programs within an organization unless the organiza- 
tion is' at a stage of maturity sufficient to foster a sophis- 
ticated understanding- of the potential of valid tests for con- 
tributing to a long-range manpower strategy. Organizations are 
^indeed looking for good selection procedures , that satisfy' their 
needs for face validity, predictive validity, and fairness, and 
would adopt such^ techniques if they were generally available. 
A major block toward adopting such devices is the perceived 
need to validate them in each user's organizations in order 
that they are job-related. Little incentive exists for 
organization to assimilate the latest and most valid selection 
practices. Recent advances in selection, documented in aca- 
demic journals, do not fit the heed of organizations which must 
still take the extra step to insure that a Piiw selection pro- • 
cedure or device will be both valid and fair in its intended 
application. The relative costs of valdation for ah 
organization are high in terms of both time and resources, 
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presenting 4 further obstacle to widespread validation within 
organ izat ions • 

Future policy regarding valid nondiscriminatory s;election 
practice should therefore encourage research by creating in- 
centives for developing new procedures and techniques in conr 
junction with specialists in selection and competency measure- 
ment. Cooperative validation projects could be supported in 
which organizations would work together with employee selection 
experts to develop reliable, valid and fair measures of compe- 
tencies required for successful job performance* Indeed much 
work is currently being conducted in the insurance and oil 
industries/ funded by a number of separate companies and in- 
volving outside experts to carry out just this kind of investi- 
gation* The incentives, for participating organizations would 
be minimal assessment development costs / and later unrestricted 
^ use of the most workable procedures. The benefits to the fund- 
ing agencies woiild be dissemination of high quality research 
practices to a larger set of organizations and more movement 
toward using job related standards for fair and equal treat- 
ment of qualified job applicants. 

Implications for Educators, .Employers, and Policymakers 
Implications for Educators 

Oh the basis of the literature on different selection tech- 
niques, a comparison of techniques, and our survey of selection 
practices in the field, we can draw certain implications . for 
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education. For example, employers have grown skeptical of the 
utility of paper and pencil tests for identifying competent job 
applicants. While test reliability may be strong/ the only 
moderate validity of tests, the test anxiety of applicants, 
skepticism about face validity, and corporate anxiety over the 
need to validate tests have led to a down turn in their use 
- since 1972"; The result is that student job applicants are less 
likely to take a selection test than they are to go through an 
interview or other selection process which they haven't exper- 
ienced in school. Two of the most common selection techniques 
are the interview and the application blank,' neither of which 
taps job-related skills by taking any measurements. Both me- 
thods are essentially self-report indicators of skills. An 
applicant may say he or she knows how to operate a drill press 
but only a-v-job sample, skill test or cneck' with a former em- 
ployer can verify that report. Ah interviewer can assess 
intelligence and interpersonal skills in the employment inter- 
view (with or without using rating scales to indicate his or 
her judgments) , and can also obtain a crude reading oh liter- 
acy, ability to follow directions, and neatness "from the appli- 
cation blank. Seldom, however, are these skills the focus of 
student preparation for a career. 

The preceding discussion leads us to the simple conclusion 
that the skills which play a critical role in surviving the 
selection process are largely unlike the skills heeded to do 
the job, nor are they skills in which most students have had 
any significant preparation. An implication for educators. 
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therefore, is that students need to prepare for the special 
demands of the selection process. The job applicant who is- 
best prepared to handle a new job is often ignored in favor of 
one who is more skillful in responding to the questions of an 
interviewer. As we have seen, even those jobs requiring only 
psychomotor skills (contrasted with intellectual or inter- 
personal jskills) invariably require "ah interview, a technique 
which is strongest in identifying interpersonal skills. Edu- 
cational institutions would do well to make their students 
aware of, or better, to educate their students in, the skills 
required to survive the job search. Further, as each selection 
method has the potential to provide some unique information 
about the applicant, educators should also prepare students to 
take advantage of each method as a way to convey information 
about themselves. For example, a neat application blank, a 
well organized and confident resume^ and a relaxed but atten- 
tive manner in an interview all convey something extra beyond 
the written or spoken word. Students who have learned to 
understand these different methods make a better impression on 
the person charged with the selection decision. 

What else can employers tell educators that will help them 
better prepare students for the world of work? For one thing , 
educators could avail themselves of analyses of ^ jobs that 
Students are likely to enter upon graduation. However, we 
should be awar^; that task analyses, the most common form of 
job analysis, can supply only a very limited account of com- 
petence. This is to say that the skills required to perform 
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job tasks satisfactorily are often quite different from the 
competencies that enable one to perform the whole 30b well. A- 
nurse, for example/ needs to acquire many task-specific 
technical skills to perform at a level of basic 'competence, but 
the competencies that supervisors and patients value, such as 
personal warmth, ^^^^^^^9 cool under pressure, and being able to 
handle a number of tasks simqitaneo^sly and efficiently, are 
not likely to. emerge from a task analysis. The example of the 
nurse is not different from, most other complex jobs in that, 
although task' analyses can be worded to derive readily the 
minimal competencies (compe^tencies needed for survival) , task 
analy^ses do not lend themselves well to identifying^ optimal 
competencies (competencies related to excellence) • Viewed in 
this light, task analyses may not serve the educator well, land 
schools which teach only to tasks to.be performed ^in jobs 
trivialize the' notion of competence. 

Competency identification through task analysis is also a 
two-^ '^ed sword for employers. We have found that relatively' 

- - L 

few organizations have condibcted a&equate job ^analyses on the 
basis of which selection strategies are chosen, even. though the 
courts have ruled against selection practices that have-not 
been suggested by such analyses.; But it seems that employers 
have come to an intuitive realization that job analysis, and 
task analysis in particular, does not provide evidence for the 
qualities they would like to see in Applicants such as integ- 
rity, interpersonal skill, intelligence, and initiative., to 
name just a few ill-defined characteristics. All evidence • 



shows that employers^ through the use of a variety of selection 
devices arid their own decision-making process/ are trying to 
select for optimal competencies as. well as minimal ones. 
•Unfortunately, organizations do not think in terms. of compe- 
tencies: They are seldom able to articulate just what these 
important performer characteristics are so that educators can 
teach them. And, as we have seen, employers are mediocre, at 
best, in making selection decisions based on these elusive 
competencies. In the general absence of information/ all that 
educational institutions can do, at present, is guess at many 
of the competencies that are implied" by job analyses. 

' The reliance^non employers for guidance in curriculum 
design, however, can be carried to an undesirable extreme. 
The goals of a liberal education must not be confused with the 
goals of eSployment. Imparting job related skills is dif- 
ferent from enabling an individual to be a more constructive 
participaxit in s^ociety* Current practices of employee selec- 
tion are nevertheless driving educational institutions to 
provide"^ training more closely related to the demands of the 
job, even though there is no assurance that the qualities on 
the basis of which one is selected into an organization are at 
all redeeming in terms of long-term job or career effective- 
ness. Although educators could learn from available job analy- 
ses, they would 'do better tb focus on generic levels o^ compe- 
tency which subsume-, job-related ' specif ics. If more direct 
translation between job tasks and a program of instruction were 
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carried out^ the result would be a technical school approach 
to education which , although a worthy alternative for some 
purposes, would do little to forward the goal of creating a 
greater awareness of one's environment and development of 
skills that help one adapt to changing situations. 




It is our conclusrqn thfatM with the exception of 
professional and vocationai^=^=^§^pols> employers are generally 
better prepared to impart specific job-related skills, and 
educational institutions are better prepared to address 
themselves to generic competencies, indeed, employers, recog- 
nizing the limitations of entry selection practices* are 
beginning to commit more resources to training and develop- 
ment of people already on the job than to selecting. people in 
advance for the right combinations of skills. Employers can- 
not realistically expect new employees to come to them fully 
prepared in terms of the skills required for their first jobs. 
Within many organizations there exists a current trend away 
from selection and toward training and career development as a 
way of acquiring individuals who possess the important- compe- 
tencies for carrying out the work of the organization. This 
is certainly a costly method of gaining individuals with the 
appropriate mix of skills, knowledges and abilities, but it may 
be more efficient in the long run than expe'cting those combi- 
nations of competencies to-be available in a job applicant 
regardless of previous educational experience. Ther.efore, 
it appears that educational institutions will be asited more 
and more to emphasize the development of generic, optimal 
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competehci(BS. Although educators should be aware of the 
requifemehts of the world of wbrk, they would do themselves 
and their students an ultimate disservice by looking to selec- 
tion criteria as the only standards against which student 
performance is to be measured. 

Implications for - Employers 

Current trends in the use of selection techniques, the t^^pe 
of data that these techniques are likely to yield, the reticence 
of organizations to carry out predictive validation studies and 
the increased emphasis,, oh training in the job suggest several 
implications for employ^ing organizations. Employers clearly 
need to begin undertaking job analyses before implementing any 
selection screening EProcess. They should begin systematic, 
empirical dqcumehtation of what it takes to do jobs well, 
rather than the prevalent armchair-theoretic approach to„ job 
analysis. Then, (documented or undocumented, employers should 
have a better chance \6f choosing selection techniques which are 

appropriate to gathering information on desired job qualities* 

_ ._ \ _ _.. _._ 

This is a first step to ascertaining whether hiring decisions 

made through the use of a given selection program are related 

to performance in critical job functions. 

Employers also need to become more aware of the systematic 

biases that are introduced into the decision making process by 

using selection devices that measure factors that are inapprop- 

riate arid unrelated to job requirements. Employers as a rule 



tend to 'Overselect for interpersonal skills and general intel- 
ligence, characteristics which have great face validity but may 
6t have any positive relationship to 30b performance. General 
intelligencer specifically, has been called into question by 
McClelland (1973) , who- noted that this construct appears hot to 
correlate with job performance within career areasr including 
those professions that aippear to demand a high degree of intel- 
lectual competence. Systematic job analyses will hot, in them- 
selves, insure that these biases will be overcome and given the 
residual unreliability of even the best selection devices admin- 
istered under controlled conditions, it is unlikely that these 
biases will be completely eradicated. 

Employers should also recognize that new employees will 
have many opportunities to train the workers for specific pos- 
itions once they are on the job. The task of selection, then, 

---- __ _ _ ___ , 

becomes more one of identifying heeded competencies that are 

unlikely to be developed oh the job rather than trying to 
account for all competencies needed to do the job at entry. A 
possible outcome of this- strategy is providing greater access 
to jobs by hohtraditiohal applicants, including women and 
minorities. Under the state of the practice, previous exper- 
ience in similar jobs is a powerful factor that determines 
whether a person will ba hired; to cite an example, sales is 
one area where previous experience weighs in favor of the 
applicant, regardless of the competence demonstrated by the 
applicant in previous employment experience. Some of the 



minimal competencies required (e^g^ product knowledge and 
knowledge of sales procedures) can be learned ideally on the 
job? while some of the more critical optimal competencies - 
(e.g^f achievement orientation and influence skills) are harder 
to develop. Employers under ?aost current selection systems 
over^select for the more easily acquired minimal competencies 
at entry/ placing at a disadvantage womehf mihoriti^Sf and 
others who have been denied equal access to such jobs in the 
past* 

In theory/ there is less likelihood that previously 
excluded applicant populations possess many optimal compe^ 
tencies to a lesser extent than do the currently favored popu- 
lation. Supporting evidence comes from the experience of the 
first author in implementing a competency-based sales selection 
system in a Fortune 500" company. Under the new system/ which 
fodused exclusively oh selecting for optimal competencies/ 
more women and fewer people with prior sales experience were 
accepted than was the case under the old system; nevertheless 
the sales generated by the hew applicant group were signfi- 
cantly greater compared to previous groups/ with less than half 
the employee turnover rate. Not only has the competency-based 
selection strategy provided greater benefit to the employer/ 
but it has also recognized the distinction between competencies 
that are developed on the job and those that are needed at en- 
try/ while providing greater opportunities for members of appli 
cant subgroups that have been denied access to jobs in the past 
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A clear recommendation is that employers and vocational 
--.organizations seek to identify the competencies required for 
the performance of job tasks and then to choose or develop 
appropriate selection techniques. The competency identifi- 
cation process is not the same as a job analysis. A job yields 
a listing of general tasks to be performed but does not, in 
itself, reveal anything about the knowledges, abilities, or 
other characteristics of the person who performs the tasks 
well. Rather than to repeat the mistakes of the majority of 
employers who take only a face validity-based approach to the 
inference of skills from the job tasks, the most logical 
approach is to identify, first, those results that a are taken 
as evidence of satisfactory or outstanding job performance, and 
then to discover the competencies that are possessed to a 
greater degree by the satisfactory or outstanding performer 
than by the less-than-satisf actory job incumbent. Gompetehcy 
"models" of the good performer in the job would supplant job 
description as the driving force in employment selection. Spec- 
ifying the most critical job tasks that are performed, and the 
competencies heeded to perform the whole job well, would go a 
long way toward in4)roving the utility and validity of selection 
systems* Additionally, this information would be of enormous 

benefit to iducators who find current job classifications and 

\ _ ^ _ 

task analyses . largely useless in improving a curriculum. 

The long-term benefit of competency-based selection to 
employers extend beyond ensuring that hew hires will be able to 
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perform their jobs adequately. Properly designed, such sielec- 
tion systems would identify training needs for new employees in 
addition to applicants' suitability for hire. A key short- 
coming of selection systems that rely oh biogr apical data and 
personality testing is that it is not clear what the individual 
can do to develop the heeded characteristics # since these sourc 
es of data rely heavily on things the person has reported doing 
in the past. By contrast/ the direct measurement of key compe- 
tencies tells the employer in a straightforward way if the 
applicant displays the characteristics associated with success 
and where the skill gaps are that can be developed- Olaereforer 
this new approach to selection is more likely to indicate 
training needs rather than fixed characteristics. 

I mplications for-^^iicymaic*jrs 

The public endorsement of equal employment opoortunity has 
been received and implemerited in ways which imply two different 
public^oHcy-goalSiT- /Sie-^iirs^ and-Tnoit common" l^tefpreatioh- 
of this endorsement is to see public policy as encouraging and 
enforcing employment parity, or employing equal or proportional 
numbers of majority and minority group members. The second 
interpretation of EEO policy is to :see it as encouraging pro- 
ductivity by selecting employees with job-related skills regard 
less of workers' group status • Both of these policy positions 
are implied by the often conflicting messages of the courts and 
enforcement agencies. Which is the intended policy? Once the 
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Federal Government makes a policy choice between employment 
par-ity and productivity ^ different action steps exist for reach 
irig either of these goals* 

The interpretation that the primary goal of pub^ ic policy 

__ _ . . ._ __i 

is to encourage employment parity is not congruent with what 

Justice Burger identified as the intent of Title VII of the 

Givil Rights Act. When he wrote the landmark Griggs decision^ 

( Willie B. Grxoas^-et al, v. Duke Power Co. Supreme Court Case 

124# October term^ 1970. Opinion delivered March 8r 1971) ^ he 

said: 

"Nothing in the Act precludes the use of testing or 
measuring procedures; obviously they are useful. What 
Congress has- forbidden is giving these devices and mechan- 
isms controlling force unless they are demonstrably a 
reasonable measure of job performance* Congress has hot 
commanded that the less qualified be preferred over the 
better qualified simply because of minority origins • Far 
from disparaging job qualifications as such# Congress has 
made such 'qualifications the controlling factor, so that 
race, religion/ nationality, and sex become irrelevant. 
What Congress has commanded is that any tests used must 
measure the person for the job and hot the person in the 
abstract." 

Unfortunately, and not surprisingly^ the public, including 
many einplbyers, have c^^^ of equa3 employment oppor- 

tunity in terms of rigid affirmative action goals, consent 
decrees with minor icy quotas, and the four -fifths (4/5) rule 
for identifying job discrimination (as described in the 1978 
Uniform Guidelines) . Employers suspect that compliance of fie- 
ei are vigilantes eager to punish businesses that don't "have 
their numbers up." They therefore protect themselves by 



hiring members of protected classes (iie., individuals identi- 
fiable on the basis of race, creed, colors sex, national ori- 
gin# ager handicap/ marital status) . This has often resulted 
in resentment by employers who feel that they have been pres- 
sured to hire individuals on the bases unrelated to job 
skills. In essence, this trend results in selection that is 
not performance based. Resources are devoted to documenting 
demographics of the work force. Thus the current practice of 
using federal compliance officers for monitoring of organiza- 
tions does hot contribute to improved job-related selection (as 
identified by Burger) , but rather to employment parity -which 
could as easily be accomplished by an unbiased lottery! If 
simplistic employment parity is the government's policy goal, 
then current mechanisms should continue to operate unchanged. 

Oh the other hand, if the intent of Congress and the public 
is to encourage job-related and unbiased selecv.ion, then the 
policy goal really is one of productivity. Job-related selec- 
tion is related to productivity in the sense that improved 
performance-based selection can upgrade skills of a work force, 
reduce heeded training resources, decrease" attrition by workers 
who don't fit the job, and reduce the expense of replacing 
incompetent workers. Prescreening for coapetence to do e- job 
is cost effective in terms of the reduction it effects on those 
later, online expenses. These kinds of improvements or pro- 
ductivity gains are obvioasly of value to all employers and 
thus ultimately to the nation. The Cdiammerce Department, 
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Government Accounting Office, and a cabinet level committee 
have all devoted time and resources to the issue of pro- 
ductivity. In a 1979 report to the President by the Council 
of Economic Advisors, the productivity issue arose as an 
important concern for government and policy i As described 
above, fair employment practice laws and validation of com- 
petency measures for selection have great potential to .contri- 
bute to a policy of encouraging productivity, but as currently 
understood, enforced and practiced, they are presently not . 
contributing much. Current enforcement has had the effect of 
organizations resorting to employmeht parity strategies, these 
organizations that have chosen ^to implement writer ion-relatod • 
validation being in the- distinct minority. Clearly new strat- 
egies need to be adopted if public policy is to encourage 
productivity through fair employment practices .and selection 
validation. 

One inexpensive procedure for improving productivity is 
to elucidate the policy of validating competence measures by 
preparing and distributing an informational pamphlet to 
employers. The return on investment for putting dollars into 
validation of selection measures is real. Examples exist of 
organizations greatly reducing turnov:»r ccsts with valid 
selection strategies. For thos^ organizations which are 
too small in size or unwilling to devote resources heeded, 
cooperative studies and validity generalization studies are 
low-cost options. While the potential impact o-f this 
information ^sharing is unknown, -more employers might be 



encouraged to validate selection procedures if validation were 
phrased not in terms of public policy and social justice, but 
father in terms of the employers' self-interest with regard to 
productivity, return oh investment, and the bottom line. 

More active efforts could exist to encourage validation of 
competency measures to use for selection. For example, it is 
hot unreasonable to think of job competency test validation as 
an investment for which the Internal Revenue Service could give 
credit. Currently, all personnel system efforts are business 
expenses, but documentation of applicant flow is not likely to 
improve an organization's productivity. While efforts to meet 
the four-fifths (4/5) rule are not investments, money and man- 
power resources put into design and validation of a selection 
measure of job competence is ah investment in improving the 
business's productivity. Perhaps some system of allowing 
investment tax credits for rigorous design and validation of 
selection strategies could be an incentive for increasing 
validation (and ultimately productivity) . 

Small companies without sufficient manpower, expertise or 
resources for the level of validation effort necessary to earn 
an investment tax credit could be encouraged to take advantage 
of other options. For example, organizations with limited 
resources might be able to qualify for sraller credits pro- 
vided they invest in and cooperate in a study of similar jobs 
within one industry, across companies. Cooperative studies 
for validity generalization is the trend of the future." 
Organizations with small numbers of jobs can band together with 



other organizations, establish the similarity of their jobs, 
statistically correct for restriction of range and other sample 
biases r and thereby establish the validity of a procedure to be 
used for selecting competent applicants* Small businesses who 
participate in such cooperative efforts might be enabled to do 
SQ with the assistance of low-interest SBA loans. This is 
another possible incentive which could persuade organizations 
to invest in validation. 

As already suggested, using federal compliance officers to 
monitor organizations with high visibility (to government, pub- 
lic or consumer) is not sufficient to bring about validation of 
selection systems. Information, appeals to the "bottom line" 
logic of managers, and financial incentives may facilitate more 
validation. Another possibility would involve efforts to up- 
grade the expertise of corporate personnel staff in the area 
of test design and validation. As one study revealed, the 
level of corporate expertise in these areas seems to be re- 
lated directly to the amount of effort organizations put into 
validation. While organizations without personnel selection or 
measurement expertise may hire external consultants and finance 
selection research and development, one way to increase the 
number or organizations validating might be to build their own 
staffs. For- example, the Federal government could mandate or 
finance fellowship programs to proviae personnel managers or 
corporate decision makers with ah opprortuhity to develop some 
knowledge off or expertise in design, validation and use of 
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selection measures. On the basis o£ that knowledge, they could 
then direct their own organizations' in these areas or make a 
more informed choice to employ qualified external consultants. 

When co nsider ing policy options and the wisdom of valid- 
ating, it is important to remember that an organization which 
chooses to aim for productivity and job-related, validated 
selection, is also choosing methods which can be used to detect 
and eliminate adverse impact, as indicated by the earlier di- 
scussion of differential validity and test bias. Organizations 
which are primarily concerned with equitable unbiased selection 
of previously disadvantaged groups might best serve that goal 
by vigorous design and validation of competency measures for 
selection. At the same time they will increase ^he' probability 
of hiring a competent, productive work force. 
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